diff --git a/analysis/characterization/elastic_migration_v2/README.md b/analysis/characterization/elastic_migration_v2/README.md
index 019ac48..d3f9709 100644
--- a/analysis/characterization/elastic_migration_v2/README.md
+++ b/analysis/characterization/elastic_migration_v2/README.md
@@ -9,82 +9,127 @@ Model: Qwen3-Coder-30B-A3B-Instruct, 8 × TP1 on H20
 This section explores whether the **B2-confirmed same-worker
 prefill–decode interference** can be relieved by selectively
 migrating prefill to a different worker for the requests where the
-interference cost would dominate the transfer cost. We implement two
-flavors of the policy (strict gates, then relaxed gates) and a clean
-isolation control (`unified_kv_both`: same picker as `unified`, but
-the vLLMs are launched in `kv_role=kv_both` so the Mooncake
-substrate is on but never triggers).
+interference cost would dominate the transfer cost. We implement
+two flavors of the routing policy (strict gates, then relaxed
+gates) and **two isolation controls** that use the unified picker
+but launch vLLMs in `kv_role=kv_both` so the connector substrate
+is on but never PD-seps:
 
-Three findings:
+- `unified_kv_both`: with **MooncakeConnector**
+- `unified_nixl_both`: with **NixlConnector** (NVIDIA's official
+  v1 connector; isolates connector implementation from policy)
 
-1. **`kv_role=kv_both` alone imposes a heavy always-on tax**: TTFT
-   p90 +45%, TPOT p90 +25%, hotspot index +19% vs plain `unified`,
-   with no PD-sep ever firing.
-2. **PD-sep almost never triggers on a real agentic workload**:
+Four findings:
+
+1. **`kv_role=kv_both` imposes a substantial always-on tax even
+   when no PD-sep ever fires**: with Mooncake it's TTFT p90 +45%,
+   TPOT p90 +25%, hotspot +19%; with NIXL it's TTFT p90 +38%,
+   TPOT p90 +16%, hotspot +0.2%.
+2. **About half of the substrate cost is generic v1-connector
+   framework overhead** (proxied by NIXL since it's the leanest
+   implementation): KV buffer GPU memory cut from the model's
+   working budget, `SchedulerOutput.kv_connector_metadata`
+   round-trip, and altered `kv_cache_manager` block-lifecycle
+   semantics. **NIXL is meaningfully better than Mooncake** but
+   still imposes a 16-38% tax vs no connector.
+3. **PD-sep almost never triggers on a real agentic workload**:
    0.16% with strict gates, 0.41% with relaxed gates. Agentic
-   workloads have 93% intra-session reuse, so most requests land on
-   workers that already hold cache — the uncached tail is too small
-   to be worth migrating.
-3. **When PD-sep does fire, the cost model is wrong by ~10–20×**:
+   workloads have 93% intra-session reuse, so most requests land
+   on workers that already hold cache — the uncached tail is too
+   small to be worth migrating.
+4. **When PD-sep does fire, the cost model is wrong by ~10–20×**:
    the calibrated `0.3s + bytes / 2.7 GB/s` predicts 1–2 s migrate
-   cost; observed TTFT on triggered requests is 12–45 s. The same
-   D-side block-reservation pressure and absence of layerwise
-   pipelining that the E2 audit flagged still dominate.
+   cost; observed TTFT on triggered requests is 12–45 s.
 
 The net latency of `unified_v2` is **not better than plain
-`unified`**. Improving agentic PD-sep requires fixing the underlying
-Mooncake transfer mechanism (E2 patches 6.1 lazy block reservation
-and 6.3 layerwise pipelining), not the routing decision.
+`unified`** under either Mooncake or NIXL substrate. Improving
+agentic PD-sep requires (a) using the leaner connector (NIXL >
+Mooncake by 5-19 pp across metrics), and (b) fixing the underlying
+transfer mechanism (E2 patches 6.1 lazy block reservation and 6.3
+layerwise pipelining), not just the routing decision.
 
 ## Substrate
 
-We compare three policies on identical traces:
+We compare four policies on identical traces:
 
 | policy | picker | vLLM launch mode | what's it for |
 |---|---|---|---|
-| `unified` | hybrid affinity + LMetric | plain (no Mooncake) | the headline baseline |
-| `unified_kv_both` | same as `unified` | `kv_role=kv_both` + bootstrap | isolation control: how much does kv_both *alone* cost? |
-| `unified_v2` | unified + selective PD-sep | `kv_role=kv_both` + bootstrap | the actual experiment |
+| `unified` | hybrid affinity + LMetric | plain (no connector) | the headline baseline |
+| `unified_kv_both` | same as `unified` | `MooncakeConnector` + `kv_both` | substrate control: Mooncake cost without PD-sep |
+| `unified_nixl_both` | same as `unified` | `NixlConnector` + `kv_both` | substrate control: NIXL cost without PD-sep, attributes overhead to "framework vs Mooncake" |
+| `unified_v2` | unified + selective PD-sep | `MooncakeConnector` + `kv_both` + bootstrap | the actual experiment |
 
-All three use the same trace, the same 8-instance topology, the same
+All four use the same trace, the same 8-instance topology, the same
 shadow-drift–corrected proxy (`scripts/cache_aware_proxy.py` post-fix
 `95c8ef8`). Plain `unified` was rerun on the patched proxy
 (`b3_sweep_20260525_095043/unified`) under the same conditions.
 
-## Result 1 — kv_both is expensive by itself
+NIXL required two launch fixes beyond Mooncake:
+- `VLLM_NIXL_SIDE_CHANNEL_PORT` must be unique per instance
+  (default 5600 → 5600..5607); otherwise instances 2..8 silently
+  hang in `zmq.error.ZMQError: Address already in use`.
+- Health-check timeout had to be raised from 180 s to 360 s
+  because NIXL initialization (UCX agent + memory registration)
+  takes ~100-150 s per instance under 8-way concurrent launch.
+
+## Result 1 — kv_both is expensive by itself, and only partly Mooncake's fault
 
 ![](figures/fig_kv_both_overhead.png)
 
 Switching the vLLM launch from plain to `kv_role=kv_both` without
-ever triggering PD-sep already costs:
+ever triggering PD-sep imposes a substrate tax. We compare the two
+connectors available in vendored vLLM:
 
-| metric | plain `unified` | `unified_kv_both` | Δ |
-|---|---:|---:|---|
-| TTFT p50 | 0.50 s | 0.50 s | +0% |
-| TTFT p90 | 7.35 s | 10.67 s | **+45%** |
-| TTFT p99 | 42.34 s | 45.19 s | +7% |
-| TPOT p90 | 17.1 ms | 21.3 ms | **+25%** |
-| E2E p90 | 18.03 s | 22.89 s | **+27%** |
-| APC | 79.4% | 78.3% | −1.1 pp |
-| hotspot index | 3.667 | **4.363** | **+19%** |
+| metric | plain `unified` | `unified_nixl_both` | `unified_kv_both` (Mooncake) |
+|---|---:|---:|---:|
+| TTFT p50 | 0.50 s | 0.51 s (+1%) | 0.50 s (+0%) |
+| **TTFT p90** | 7.35 s | **10.13 s (+38%)** | **10.67 s (+45%)** |
+| TTFT p99 | 42.34 s | 44.58 s (+5%) | 45.19 s (+7%) |
+| TPOT p90 | 17.1 ms | **19.8 ms (+16%)** | **21.3 ms (+25%)** |
+| E2E p90 | 18.03 s | **21.18 s (+17%)** | **22.89 s (+27%)** |
+| APC | 79.4% | 79.1% (−0.3 pp) | 78.3% (−1.1 pp) |
+| **hotspot index** | 3.667 | **3.674 (+0.2%)** | **4.363 (+19%)** |
+| interference index | n/a | 5.58 | 8.57 |
 
-Two contributing factors:
+![](figures/fig_connector_substrate_attribution.png)
 
-1. **The Mooncake `MooncakeConnector` runs even when no transfer is
-   pending.** Every scheduler step it walks `set(cache.keys())`
-   against `_known_hash_keys` (E2 audit §6.5) and updates the
-   `KVConnectorMetadata`. This is O(|cache|) per step on every
-   engine, even when no producer/consumer relationship is active.
-2. **Block reservation semantics differ** under kv_both. The
-   scheduler treats blocks as candidates for export-to-others, so
-   the prefix cache LRU pressure is slightly different (we lose 1
-   pp APC).
+Reading the table from left to right gives a clean attribution:
 
-Practical implication: **you don't enable kv_both for free**. If a
-deployment wants the option to do PD-sep selectively, the 45% TTFT
-p90 tax applies even on requests that stay local. This needs to
-recoverable cost before any selective-PD-sep policy is worth
-shipping.
+- **NIXL−plain** = the **v1-connector framework's irreducible cost**
+  (TTFT p90 +38%, TPOT p90 +16%, E2E p90 +17%). This is the cost
+  *any* v1 KV connector imposes:
+  - the 1 GB `kv_buffer_size` carved from `gpu-memory-utilization`,
+    reducing the KV cache budget;
+  - per-step `SchedulerOutput.kv_connector_metadata` serialization
+    and round-trip through the connector worker;
+  - altered block-lifecycle semantics in `kv_cache_manager`
+    (`delay_free_blocks=True` is the default once any connector is
+    loaded, slowing LRU eviction).
+- **Mooncake−NIXL** = the **Mooncake-implementation-specific extra**
+  (TTFT p90 +7 pp, TPOT p90 +9 pp, E2E p90 +10 pp, hotspot +19 pp).
+  This is the cost Mooncake's design choices add on top of the
+  generic framework:
+  - per-scheduler-step `set(self._block_pool.cache.keys())` diff
+    against `_known_hash_keys` (`mooncake_connector.py:432-456`)
+    walks O(|cache|) on every step on every engine, costing ~4 M
+    set operations per second on a 200 k-block cache;
+  - the hash sync runs even when no `direct_read` consumer is
+    present, so the cost is paid unconditionally;
+  - block-lifecycle is further constrained because Mooncake
+    requires `delay_free` until the explicit `finished_sending`
+    arrives, vs NIXL which can release blocks earlier.
+
+The **most striking gap is hotspot**: Mooncake's per-step hash
+sync runs on the scheduler's GIL and disrupts the timeliness of
+routing decisions, amplifying load imbalance by 19%. NIXL has no
+equivalent global-state maintenance and preserves the plain-unified
+hotspot to within 0.2%.
+
+Practical implication: **you don't enable any v1 KV connector for
+free**, but if you have to enable one, NIXL is meaningfully cheaper
+than Mooncake. Even NIXL's 38% TTFT p90 tax is large enough that
+PD-sep needs to recover it on a non-trivial fraction of requests
+before being worth it.
 
 ## Result 2 — PD-sep rarely fires on a real agentic trace
 
@@ -153,24 +198,24 @@ The first-token clock for the 49 k request is **21× the model's
 prediction**. This is not a small mis-tuning — it's a structurally
 different model.
 
-## Result 4 — three-way comparison
+## Result 4 — four-way comparison
 
 ![](figures/fig_three_way_hotspot.png)
 
 The full table:
 
-| metric | unified (plain) | unified_kv_both | unified_v2 (relaxed) |
-|---|---:|---:|---:|
-| n_ok | 1214 | 1214 | 1214 |
-| TTFT p50 | 0.50 s | 0.50 s | 0.49 s |
-| TTFT p90 | 7.35 s | 10.67 s | 10.98 s |
-| TTFT p99 | 42.34 s | 45.19 s | 49.45 s |
-| TPOT p90 | 17.1 ms | 21.3 ms | 18.4 ms |
-| E2E p90 | 18.03 s | 22.89 s | 22.53 s |
-| APC | 79.4% | 78.3% | 77.6% |
-| interference index | n/a (no engine_state) | 8.57 | 8.46 |
-| hotspot index | 3.667 | 4.363 | 3.910 |
-| n_slow | 189 | 198 | 198 |
+| metric | unified (plain) | unified_nixl_both | unified_kv_both (Mooncake) | unified_v2 (relaxed) |
+|---|---:|---:|---:|---:|
+| n_ok | 1214 | 1214 | 1214 | 1214 |
+| TTFT p50 | 0.50 s | 0.51 s | 0.50 s | 0.49 s |
+| TTFT p90 | 7.35 s | 10.13 s | 10.67 s | 10.98 s |
+| TTFT p99 | 42.34 s | 44.58 s | 45.19 s | 49.45 s |
+| TPOT p90 | 17.1 ms | 19.8 ms | 21.3 ms | 18.4 ms |
+| E2E p90 | 18.03 s | 21.18 s | 22.89 s | 22.53 s |
+| APC | 79.4% | 79.1% | 78.3% | 77.6% |
+| interference index | n/a | 5.58 | 8.57 | 8.46 |
+| hotspot index | 3.667 | 3.674 | 4.363 | 3.910 |
+| n_slow | 189 | 192 | 198 | 198 |
 
 ### v2 vs the kv_both control (the right comparison)
 
diff --git a/analysis/characterization/elastic_migration_v2/data/b3_policy_comparison.json b/analysis/characterization/elastic_migration_v2/data/b3_policy_comparison.json
index d301947..41f3c3c 100644
--- a/analysis/characterization/elastic_migration_v2/data/b3_policy_comparison.json
+++ b/analysis/characterization/elastic_migration_v2/data/b3_policy_comparison.json
@@ -155,6 +155,32 @@
         "unknown": 49
       }
     },
+    {
+      "policy": "unified_nixl_both",
+      "n_ok": 1214,
+      "n_total": 1214,
+      "ttft_p50_s": 0.5138550130068325,
+      "ttft_p90_s": 10.127110345300755,
+      "ttft_p99_s": 44.5789094621703,
+      "tpot_p50_s": 0.008423213202440761,
+      "tpot_p90_s": 0.019759515867947428,
+      "tpot_p99_s": 0.1079433335279151,
+      "e2e_p50_s": 1.866590676479973,
+      "e2e_p90_s": 21.179128799570027,
+      "e2e_p99_s": 96.01196486203865,
+      "apc_ratio": 0.791441828164218,
+      "interference_index": 5.580715970433481,
+      "hotspot_index_ttft_p90": 3.673957447190547,
+      "reuse_intra_frac": 0.930632797070364,
+      "reuse_cross_frac": 0.05718149217603143,
+      "n_slow": 192,
+      "failure_counts": {
+        "cache_miss_large_append": 21,
+        "hot_worker_queue": 75,
+        "same_worker_prefill_overlap": 72,
+        "unknown": 24
+      }
+    },
     {
       "policy": "unified_v2",
       "n_ok": 1214,
diff --git a/analysis/characterization/elastic_migration_v2/data/breakdown_unified_nixl_both.json b/analysis/characterization/elastic_migration_v2/data/breakdown_unified_nixl_both.json
new file mode 100644
index 0000000..67078be
--- /dev/null
+++ b/analysis/characterization/elastic_migration_v2/data/breakdown_unified_nixl_both.json
@@ -0,0 +1 @@
+[{"request_id":"1237198:1:1237198:0","session_id":"1237198","input_length":8228,"t_proxy_recv":348308.40859294,"t_decision_unix":1779779173.4225338,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":8228,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":8228,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":8228,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":8228,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":8228,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":8228,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":8228,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":8228,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":8228,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348309.297296933,"t_first_token_unix":1779779174.3112411,"t_done":348309.393787895,"t_done_unix":1779779174.407733},{"request_id":"1237337:1:1237337:1","session_id":"1237337","input_length":3976,"t_proxy_recv":348308.825073522,"t_decision_unix":1779779173.8390148,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":3976,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":3976,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":8228,"ongoing_decode_tokens":0,"pending_prefill_tokens":8228,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":3976,"score_linear":8228.0,"score_lmetric":12204},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":3976,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":3976,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":3976,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":3976,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":3976,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":3976,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348309.416800452,"t_first_token_unix":1779779174.4307435,"t_done":348309.809742669,"t_done_unix":1779779174.8236864},{"request_id":"1237198:2:1237773:2","session_id":"1237198","input_length":10237,"t_proxy_recv":348310.376467248,"t_decision_unix":1779779175.390408,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":8192,"affinity_cache_ratio":0.8002344436846732,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":8192,"estimated_new_tokens":2045,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":10237,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":16,"cache_hit":8192,"new_prefill":2045,"score_linear":-8192.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":10237,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":7,"cache_hit":0,"new_prefill":10237,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":10237,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":10237,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":10237,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":10237,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-8192.0,"chosen_score_lmetric":0,"t_first_token":348310.668854852,"t_first_token_unix":1779779175.6827986,"t_done":348310.80432066,"t_done_unix":1779779175.818264},{"request_id":"1237877:1:1237877:3","session_id":"1237877","input_length":749,"t_proxy_recv":348310.834225132,"t_decision_unix":1779779175.8481667,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":749,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":749,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":20,"cache_hit":0,"new_prefill":749,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":749,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":7,"cache_hit":0,"new_prefill":749,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":749,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":749,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":749,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":749,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348310.954441032,"t_first_token_unix":1779779175.9683843,"t_done":348311.082525295,"t_done_unix":1779779176.0964682},{"request_id":"1239034:1:1239034:4","session_id":"1239034","input_length":13489,"t_proxy_recv":348314.610560028,"t_decision_unix":1779779179.6245012,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":13489,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":13489,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":20,"cache_hit":0,"new_prefill":13489,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":13489,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":8,"cache_hit":0,"new_prefill":13489,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":13489,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":13489,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":13489,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":13489,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348316.106518137,"t_first_token_unix":1779779181.1204622,"t_done":348316.561479234,"t_done_unix":1779779181.5754225},{"request_id":"1237877:2:1239577:6","session_id":"1237877","input_length":1620,"t_proxy_recv":348316.495814182,"t_decision_unix":1779779181.5097554,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":512,"affinity_cache_ratio":0.3160493827160494,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":512,"estimated_new_tokens":1108,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":1620,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":20,"cache_hit":0,"new_prefill":1620,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":1620,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":8,"cache_hit":512,"new_prefill":1108,"score_linear":-512.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":13489,"ongoing_decode_tokens":13489,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":1620,"score_linear":13489.0,"score_lmetric":1620},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":1620,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":37165,"ongoing_decode_tokens":0,"pending_prefill_tokens":37165,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":1620,"score_linear":37165.0,"score_lmetric":38785},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":1620,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-512.0,"chosen_score_lmetric":0,"t_first_token":348316.578207994,"t_first_token_unix":1779779181.592151,"t_done":348317.030197005,"t_done_unix":1779779182.0441403},{"request_id":"1239704:1:1239704:7","session_id":"1239704","input_length":5194,"t_proxy_recv":348316.938209599,"t_decision_unix":1779779181.9521508,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":5194,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":5194,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":20,"cache_hit":0,"new_prefill":5194,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":5194,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":1620,"ongoing_decode_tokens":1620,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":8,"cache_hit":0,"new_prefill":5194,"score_linear":1620.0,"score_lmetric":5194},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":5194,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":5194,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":37165,"ongoing_decode_tokens":0,"pending_prefill_tokens":37165,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":5194,"score_linear":37165.0,"score_lmetric":42359},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":5194,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348317.579479258,"t_first_token_unix":1779779182.593423,"t_done":348317.851914997,"t_done_unix":1779779182.8658586},{"request_id":"1240924:1:1240924:11","session_id":"1240924","input_length":7429,"t_proxy_recv":348320.958528432,"t_decision_unix":1779779185.9724693,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":7429,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":10,"cache_hit":0,"new_prefill":7429,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":20,"cache_hit":0,"new_prefill":7429,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":66214,"ongoing_decode_tokens":0,"pending_prefill_tokens":66214,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":7429,"score_linear":66214.0,"score_lmetric":73643},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":30040,"ongoing_decode_tokens":0,"pending_prefill_tokens":30040,"num_requests":1,"active_p_offloads":0,"cached_blocks":10,"cache_hit":0,"new_prefill":7429,"score_linear":30040.0,"score_lmetric":37469},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":18420,"ongoing_decode_tokens":18420,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":7429,"score_linear":18420.0,"score_lmetric":7429},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":7429,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":37165,"ongoing_decode_tokens":37165,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":7429,"score_linear":37165.0,"score_lmetric":7429},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":7429,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348321.500044438,"t_first_token_unix":1779779186.5139883,"t_done":348321.677893728,"t_done_unix":1779779186.691837},{"request_id":"1239100:1:1239100:5","session_id":"1239100","input_length":37165,"t_proxy_recv":348314.853471183,"t_decision_unix":1779779179.8674126,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":37165,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":37165,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":20,"cache_hit":0,"new_prefill":37165,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":37165,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":8,"cache_hit":0,"new_prefill":37165,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":13489,"ongoing_decode_tokens":0,"pending_prefill_tokens":13489,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":37165,"score_linear":13489.0,"score_lmetric":50654},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":37165,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":37165,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":37165,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348320.854641671,"t_first_token_unix":1779779185.8685856,"t_done":348322.11146858,"t_done_unix":1779779187.1254125},{"request_id":"1240198:1:1240198:9","session_id":"1240198","input_length":30040,"t_proxy_recv":348318.475050263,"t_decision_unix":1779779183.4889915,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":30040,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":10,"cache_hit":0,"new_prefill":30040,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":20,"cache_hit":0,"new_prefill":30040,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":66214,"ongoing_decode_tokens":0,"pending_prefill_tokens":66214,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":30040,"score_linear":66214.0,"score_lmetric":96254},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":10,"cache_hit":0,"new_prefill":30040,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":30040,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":30040,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":37165,"ongoing_decode_tokens":0,"pending_prefill_tokens":37165,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":30040,"score_linear":37165.0,"score_lmetric":67205},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":30040,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348322.399545283,"t_first_token_unix":1779779187.4134889,"t_done":348323.049585223,"t_done_unix":1779779188.0635283},{"request_id":"1241953:1:1241953:13","session_id":"1241953","input_length":1093,"t_proxy_recv":348324.375732467,"t_decision_unix":1779779189.389674,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":1093,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":10,"cache_hit":0,"new_prefill":1093,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":34,"cache_hit":0,"new_prefill":1093,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":66214,"ongoing_decode_tokens":0,"pending_prefill_tokens":66214,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":1093,"score_linear":66214.0,"score_lmetric":67307},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":68,"cache_hit":0,"new_prefill":1093,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":18420,"ongoing_decode_tokens":18420,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":1093,"score_linear":18420.0,"score_lmetric":1093},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":1093,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":7587,"ongoing_decode_tokens":0,"pending_prefill_tokens":7587,"num_requests":1,"active_p_offloads":0,"cached_blocks":70,"cache_hit":0,"new_prefill":1093,"score_linear":7587.0,"score_lmetric":8680},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":1093,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348324.495402749,"t_first_token_unix":1779779189.509346,"t_done":348325.016096933,"t_done_unix":1779779190.0300403},{"request_id":"1241942:1:1241942:12","session_id":"1241942","input_length":7587,"t_proxy_recv":348324.330242972,"t_decision_unix":1779779189.3441844,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":7587,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":10,"cache_hit":0,"new_prefill":7587,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":34,"cache_hit":0,"new_prefill":7587,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":66214,"ongoing_decode_tokens":0,"pending_prefill_tokens":66214,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":7587,"score_linear":66214.0,"score_lmetric":73801},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":68,"cache_hit":0,"new_prefill":7587,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":18420,"ongoing_decode_tokens":18420,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":7587,"score_linear":18420.0,"score_lmetric":7587},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":7587,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":70,"cache_hit":0,"new_prefill":7587,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":7587,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348324.861773729,"t_first_token_unix":1779779189.8757176,"t_done":348325.053293468,"t_done_unix":1779779190.0672371},{"request_id":"1237877:3:1242953:15","session_id":"1237877","input_length":2659,"t_proxy_recv":348327.762381551,"t_decision_unix":1779779192.7763228,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":1536,"affinity_cache_ratio":0.5776607747273411,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":1536,"estimated_new_tokens":1123,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":17157,"ongoing_decode_tokens":0,"pending_prefill_tokens":17157,"num_requests":1,"active_p_offloads":0,"cached_blocks":10,"cache_hit":0,"new_prefill":2659,"score_linear":17157.0,"score_lmetric":19816},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":2659,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":66214,"ongoing_decode_tokens":0,"pending_prefill_tokens":66214,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":2659,"score_linear":66214.0,"score_lmetric":68873},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":68,"cache_hit":1536,"new_prefill":1123,"score_linear":-1536.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":18420,"ongoing_decode_tokens":18420,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":2659,"score_linear":18420.0,"score_lmetric":2659},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":2659,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":84,"cache_hit":0,"new_prefill":2659,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":2659,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-1536.0,"chosen_score_lmetric":0,"t_first_token":348327.856101799,"t_first_token_unix":1779779192.8700457,"t_done":348327.920213205,"t_done_unix":1779779192.9341621},{"request_id":"1242838:1:1242838:14","session_id":"1242838","input_length":17157,"t_proxy_recv":348327.382578056,"t_decision_unix":1779779192.3965192,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":17157,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":10,"cache_hit":0,"new_prefill":17157,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":17157,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":66214,"ongoing_decode_tokens":0,"pending_prefill_tokens":66214,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":17157,"score_linear":66214.0,"score_lmetric":83371},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":68,"cache_hit":0,"new_prefill":17157,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":18420,"ongoing_decode_tokens":18420,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":17157,"score_linear":18420.0,"score_lmetric":17157},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":17157,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":84,"cache_hit":0,"new_prefill":17157,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":17157,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348329.14805964,"t_first_token_unix":1779779194.1620033,"t_done":348330.054763071,"t_done_unix":1779779195.0687065},{"request_id":"1239704:2:1243943:17","session_id":"1239704","input_length":14812,"t_proxy_recv":348331.043074785,"t_decision_unix":1779779196.057016,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":5120,"affinity_cache_ratio":0.3456656764785309,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":5120,"estimated_new_tokens":9692,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":43,"cache_hit":5120,"new_prefill":9692,"score_linear":-5120.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":11074,"ongoing_decode_tokens":0,"pending_prefill_tokens":11074,"num_requests":1,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":14812,"score_linear":11074.0,"score_lmetric":25886},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":66214,"ongoing_decode_tokens":0,"pending_prefill_tokens":66214,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":14812,"score_linear":66214.0,"score_lmetric":81026},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":70,"cache_hit":0,"new_prefill":14812,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":18420,"ongoing_decode_tokens":18420,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":14812,"score_linear":18420.0,"score_lmetric":14812},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":14812,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":84,"cache_hit":0,"new_prefill":14812,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":14812,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-5120.0,"chosen_score_lmetric":0,"t_first_token":348332.127193242,"t_first_token_unix":1779779197.1411371,"t_done":348332.24425326,"t_done_unix":1779779197.2581968},{"request_id":"1243831:1:1243831:16","session_id":"1243831","input_length":11074,"t_proxy_recv":348330.639096543,"t_decision_unix":1779779195.653038,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":11074,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":43,"cache_hit":0,"new_prefill":11074,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":11074,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":66214,"ongoing_decode_tokens":0,"pending_prefill_tokens":66214,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":11074,"score_linear":66214.0,"score_lmetric":77288},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":70,"cache_hit":0,"new_prefill":11074,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":18420,"ongoing_decode_tokens":18420,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":11074,"score_linear":18420.0,"score_lmetric":11074},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":11074,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":84,"cache_hit":0,"new_prefill":11074,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":11074,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348331.571930332,"t_first_token_unix":1779779196.585874,"t_done":348333.043249364,"t_done_unix":1779779198.0571926},{"request_id":"1239755:1:1239755:8","session_id":"1239755","input_length":66214,"t_proxy_recv":348317.150397116,"t_decision_unix":1779779182.1643384,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":66214,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":5194,"ongoing_decode_tokens":0,"pending_prefill_tokens":5194,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":66214,"score_linear":5194.0,"score_lmetric":71408},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":20,"cache_hit":0,"new_prefill":66214,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":66214,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":10,"cache_hit":0,"new_prefill":66214,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":66214,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":66214,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":37165,"ongoing_decode_tokens":0,"pending_prefill_tokens":37165,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":66214,"score_linear":37165.0,"score_lmetric":103379},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":66214,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348333.025282558,"t_first_token_unix":1779779198.039235,"t_done":348333.718896085,"t_done_unix":1779779198.7328393},{"request_id":"1239034:2:1240558:10","session_id":"1239034","input_length":18420,"t_proxy_recv":348319.708541882,"t_decision_unix":1779779184.7224832,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":13312,"affinity_cache_ratio":0.7226927252985885,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":13312,"estimated_new_tokens":5108,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":10,"cache_hit":0,"new_prefill":18420,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":20,"cache_hit":0,"new_prefill":18420,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":66214,"ongoing_decode_tokens":0,"pending_prefill_tokens":66214,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":18420,"score_linear":66214.0,"score_lmetric":84634},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":30040,"ongoing_decode_tokens":0,"pending_prefill_tokens":30040,"num_requests":1,"active_p_offloads":0,"cached_blocks":10,"cache_hit":0,"new_prefill":18420,"score_linear":30040.0,"score_lmetric":48460},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":13312,"new_prefill":5108,"score_linear":-13312.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":18420,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":37165,"ongoing_decode_tokens":0,"pending_prefill_tokens":37165,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":18420,"score_linear":37165.0,"score_lmetric":55585},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":18420,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-13312.0,"chosen_score_lmetric":0,"t_first_token":348320.409213263,"t_first_token_unix":1779779185.4231567,"t_done":348333.85275456,"t_done_unix":1779779198.866698},{"request_id":"1244640:1:1244640:18","session_id":"1244640","input_length":8274,"t_proxy_recv":348333.534415616,"t_decision_unix":1779779198.5483565,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":8274,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":61,"cache_hit":0,"new_prefill":8274,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":58,"cache_hit":0,"new_prefill":8274,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":66214,"ongoing_decode_tokens":66214,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":8274,"score_linear":66214.0,"score_lmetric":8274},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":70,"cache_hit":0,"new_prefill":8274,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":18420,"ongoing_decode_tokens":18420,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":8274,"score_linear":18420.0,"score_lmetric":8274},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":8274,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":84,"cache_hit":0,"new_prefill":8274,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":8274,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348334.128526716,"t_first_token_unix":1779779199.1424706,"t_done":348334.333403137,"t_done_unix":1779779199.3473465},{"request_id":"1245572:1:1245572:19","session_id":"1245572","input_length":6109,"t_proxy_recv":348336.668239179,"t_decision_unix":1779779201.6821806,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":6109,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":61,"cache_hit":0,"new_prefill":6109,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":58,"cache_hit":0,"new_prefill":6109,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":129,"cache_hit":0,"new_prefill":6109,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":86,"cache_hit":0,"new_prefill":6109,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":37,"cache_hit":0,"new_prefill":6109,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":6109,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":84,"cache_hit":0,"new_prefill":6109,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":6109,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348337.370399336,"t_first_token_unix":1779779202.3843424,"t_done":348337.615709091,"t_done_unix":1779779202.6296523},{"request_id":"1241953:2:1245624:20","session_id":"1241953","input_length":10382,"t_proxy_recv":348336.758298542,"t_decision_unix":1779779201.7722402,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":1024,"affinity_cache_ratio":0.09863224812174919,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":1024,"estimated_new_tokens":9358,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":61,"cache_hit":0,"new_prefill":10382,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":58,"cache_hit":1024,"new_prefill":9358,"score_linear":-1024.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":129,"cache_hit":0,"new_prefill":10382,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":86,"cache_hit":0,"new_prefill":10382,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":37,"cache_hit":0,"new_prefill":10382,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":10382,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":84,"cache_hit":0,"new_prefill":10382,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":6109,"ongoing_decode_tokens":0,"pending_prefill_tokens":6109,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":10382,"score_linear":6109.0,"score_lmetric":16491}],"chosen_score_linear":-1024.0,"chosen_score_lmetric":0,"t_first_token":348337.52707923,"t_first_token_unix":1779779202.5410225,"t_done":348337.713412426,"t_done_unix":1779779202.727356},{"request_id":"1245572:2:1246348:22","session_id":"1245572","input_length":6182,"t_proxy_recv":348339.264382313,"t_decision_unix":1779779204.2783237,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":6144,"affinity_cache_ratio":0.9938531219670009,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":6144,"estimated_new_tokens":38,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":61,"cache_hit":0,"new_prefill":6182,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":17447,"ongoing_decode_tokens":0,"pending_prefill_tokens":10279,"num_requests":1,"active_p_offloads":0,"cached_blocks":76,"cache_hit":0,"new_prefill":6182,"score_linear":17447.0,"score_lmetric":16461},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":129,"cache_hit":0,"new_prefill":6182,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":86,"cache_hit":0,"new_prefill":6182,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":37,"cache_hit":0,"new_prefill":6182,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":6182,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":84,"cache_hit":0,"new_prefill":6182,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":12,"cache_hit":6144,"new_prefill":38,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-6144.0,"chosen_score_lmetric":0,"t_first_token":348339.293467032,"t_first_token_unix":1779779204.30741,"t_done":348339.475272336,"t_done_unix":1779779204.4892154},{"request_id":"1240924:2:1246244:21","session_id":"1240924","input_length":17447,"t_proxy_recv":348338.890418795,"t_decision_unix":1779779203.9043598,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":7168,"affinity_cache_ratio":0.4108442712214134,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":7168,"estimated_new_tokens":10279,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":61,"cache_hit":0,"new_prefill":17447,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":76,"cache_hit":7168,"new_prefill":10279,"score_linear":-7168.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":129,"cache_hit":0,"new_prefill":17447,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":86,"cache_hit":0,"new_prefill":17447,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":37,"cache_hit":0,"new_prefill":17447,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":17447,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":84,"cache_hit":0,"new_prefill":17447,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":12,"cache_hit":0,"new_prefill":17447,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-7168.0,"chosen_score_lmetric":0,"t_first_token":348340.100072059,"t_first_token_unix":1779779205.1140158,"t_done":348340.474840576,"t_done_unix":1779779205.4887836},{"request_id":"1241942:2:1247111:23","session_id":"1241942","input_length":17975,"t_proxy_recv":348341.937617481,"t_decision_unix":1779779206.9515586,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":7168,"affinity_cache_ratio":0.3987760778859527,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":7168,"estimated_new_tokens":10807,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":61,"cache_hit":0,"new_prefill":17975,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":96,"cache_hit":0,"new_prefill":17975,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":129,"cache_hit":0,"new_prefill":17975,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":86,"cache_hit":0,"new_prefill":17975,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":37,"cache_hit":0,"new_prefill":17975,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":17975,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":84,"cache_hit":7168,"new_prefill":10807,"score_linear":-7168.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":12,"cache_hit":0,"new_prefill":17975,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-7168.0,"chosen_score_lmetric":0,"t_first_token":348343.228700133,"t_first_token_unix":1779779208.242644,"t_done":348343.413647507,"t_done_unix":1779779208.427591},{"request_id":"1244640:2:1247941:24","session_id":"1244640","input_length":10696,"t_proxy_recv":348344.754745955,"t_decision_unix":1779779209.7686868,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":8192,"affinity_cache_ratio":0.7658937920718025,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":8192,"estimated_new_tokens":2504,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":61,"cache_hit":0,"new_prefill":10696,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":96,"cache_hit":0,"new_prefill":10696,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":129,"cache_hit":0,"new_prefill":10696,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":86,"cache_hit":8192,"new_prefill":2504,"score_linear":-8192.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":37,"cache_hit":0,"new_prefill":10696,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":10696,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":105,"cache_hit":0,"new_prefill":10696,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":12,"cache_hit":0,"new_prefill":10696,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-8192.0,"chosen_score_lmetric":0,"t_first_token":348345.079757226,"t_first_token_unix":1779779210.0937011,"t_done":348345.126464801,"t_done_unix":1779779210.1404078},{"request_id":"1248768:1:1248768:25","session_id":"1248768","input_length":11515,"t_proxy_recv":348347.535892747,"t_decision_unix":1779779212.549834,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":11515,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":61,"cache_hit":0,"new_prefill":11515,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":96,"cache_hit":0,"new_prefill":11515,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":129,"cache_hit":0,"new_prefill":11515,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":90,"cache_hit":0,"new_prefill":11515,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":37,"cache_hit":0,"new_prefill":11515,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":11515,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":105,"cache_hit":0,"new_prefill":11515,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":12,"cache_hit":0,"new_prefill":11515,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348348.51639578,"t_first_token_unix":1779779213.5303395,"t_done":348348.781011142,"t_done_unix":1779779213.7949543},{"request_id":"1249058:1:1249058:26","session_id":"1249058","input_length":4126,"t_proxy_recv":348348.553975439,"t_decision_unix":1779779213.567917,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":4126,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":11515,"ongoing_decode_tokens":11515,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":61,"cache_hit":0,"new_prefill":4126,"score_linear":11515.0,"score_lmetric":4126},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":96,"cache_hit":0,"new_prefill":4126,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":129,"cache_hit":0,"new_prefill":4126,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":90,"cache_hit":0,"new_prefill":4126,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":37,"cache_hit":0,"new_prefill":4126,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":4126,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":105,"cache_hit":0,"new_prefill":4126,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":12,"cache_hit":0,"new_prefill":4126,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348348.819929086,"t_first_token_unix":1779779213.833872,"t_done":348348.98636617,"t_done_unix":1779779214.000309},{"request_id":"1248768:2:1249765:27","session_id":"1248768","input_length":12188,"t_proxy_recv":348350.81746022,"t_decision_unix":1779779215.8314018,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":11264,"affinity_cache_ratio":0.924187725631769,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":11264,"estimated_new_tokens":924,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":83,"cache_hit":11264,"new_prefill":924,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":96,"cache_hit":0,"new_prefill":12188,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":129,"cache_hit":0,"new_prefill":12188,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":90,"cache_hit":0,"new_prefill":12188,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":45,"cache_hit":0,"new_prefill":12188,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":12188,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":105,"cache_hit":0,"new_prefill":12188,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":12,"cache_hit":0,"new_prefill":12188,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-11264.0,"chosen_score_lmetric":0,"t_first_token":348350.960472889,"t_first_token_unix":1779779215.974416,"t_done":348351.430870392,"t_done_unix":1779779216.4448137},{"request_id":"1249058:2:1249872:28","session_id":"1249058","input_length":7482,"t_proxy_recv":348351.209677834,"t_decision_unix":1779779216.223619,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":4096,"affinity_cache_ratio":0.5474472066292435,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":4096,"estimated_new_tokens":3386,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":12188,"ongoing_decode_tokens":12188,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":83,"cache_hit":0,"new_prefill":7482,"score_linear":12188.0,"score_lmetric":7482},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":96,"cache_hit":0,"new_prefill":7482,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":129,"cache_hit":0,"new_prefill":7482,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":90,"cache_hit":0,"new_prefill":7482,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":45,"cache_hit":4096,"new_prefill":3386,"score_linear":-4096.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":7482,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":105,"cache_hit":0,"new_prefill":7482,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":12,"cache_hit":0,"new_prefill":7482,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-4096.0,"chosen_score_lmetric":0,"t_first_token":348351.531601972,"t_first_token_unix":1779779216.545545,"t_done":348351.659485982,"t_done_unix":1779779216.6734293},{"request_id":"1242838:2:1250503:29","session_id":"1242838","input_length":21578,"t_proxy_recv":348353.485128051,"t_decision_unix":1779779218.4990695,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":16896,"affinity_cache_ratio":0.783019742330151,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":16896,"estimated_new_tokens":4682,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":84,"cache_hit":16896,"new_prefill":4682,"score_linear":-16896.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":96,"cache_hit":0,"new_prefill":21578,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":129,"cache_hit":0,"new_prefill":21578,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":90,"cache_hit":0,"new_prefill":21578,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":51,"cache_hit":0,"new_prefill":21578,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":21578,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":105,"cache_hit":0,"new_prefill":21578,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":12,"cache_hit":0,"new_prefill":21578,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-16896.0,"chosen_score_lmetric":0,"t_first_token":348354.197176021,"t_first_token_unix":1779779219.21112,"t_done":348354.692994519,"t_done_unix":1779779219.7069376},{"request_id":"1250772:1:1250772:30","session_id":"1250772","input_length":6829,"t_proxy_recv":348354.344073373,"t_decision_unix":1779779219.3580148,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":6829,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":21578,"ongoing_decode_tokens":21578,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":84,"cache_hit":0,"new_prefill":6829,"score_linear":21578.0,"score_lmetric":6829},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":96,"cache_hit":0,"new_prefill":6829,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":129,"cache_hit":0,"new_prefill":6829,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":90,"cache_hit":0,"new_prefill":6829,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":51,"cache_hit":0,"new_prefill":6829,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":6829,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":105,"cache_hit":0,"new_prefill":6829,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":12,"cache_hit":0,"new_prefill":6829,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348355.349714683,"t_first_token_unix":1779779220.3636582,"t_done":348356.947318144,"t_done_unix":1779779221.961262},{"request_id":"1251059:1:1251059:32","session_id":"1251059","input_length":16512,"t_proxy_recv":348355.526395302,"t_decision_unix":1779779220.5403368,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":16512,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":93,"cache_hit":0,"new_prefill":16512,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":96,"cache_hit":0,"new_prefill":16512,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":129,"cache_hit":0,"new_prefill":16512,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":90,"cache_hit":0,"new_prefill":16512,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":51,"cache_hit":0,"new_prefill":16512,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":6829,"ongoing_decode_tokens":6829,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":16512,"score_linear":6829.0,"score_lmetric":16512},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":20086,"ongoing_decode_tokens":0,"pending_prefill_tokens":20086,"num_requests":1,"active_p_offloads":0,"cached_blocks":105,"cache_hit":0,"new_prefill":16512,"score_linear":20086.0,"score_lmetric":36598},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":12,"cache_hit":0,"new_prefill":16512,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348357.088789851,"t_first_token_unix":1779779222.1027339,"t_done":348357.091170829,"t_done_unix":1779779222.1051137},{"request_id":"1248768:3:1251261:33","session_id":"1248768","input_length":16989,"t_proxy_recv":348356.100598398,"t_decision_unix":1779779221.1145399,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":11776,"affinity_cache_ratio":0.6931543940196597,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":11776,"estimated_new_tokens":5213,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":93,"cache_hit":11776,"new_prefill":5213,"score_linear":-11776.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":96,"cache_hit":0,"new_prefill":16989,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":16512,"ongoing_decode_tokens":0,"pending_prefill_tokens":16512,"num_requests":1,"active_p_offloads":0,"cached_blocks":129,"cache_hit":0,"new_prefill":16989,"score_linear":16512.0,"score_lmetric":33501},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":90,"cache_hit":0,"new_prefill":16989,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":51,"cache_hit":0,"new_prefill":16989,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":6829,"ongoing_decode_tokens":6829,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":16989,"score_linear":6829.0,"score_lmetric":16989},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":20086,"ongoing_decode_tokens":0,"pending_prefill_tokens":20086,"num_requests":1,"active_p_offloads":0,"cached_blocks":105,"cache_hit":0,"new_prefill":16989,"score_linear":20086.0,"score_lmetric":37075},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":12,"cache_hit":0,"new_prefill":16989,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-11776.0,"chosen_score_lmetric":0,"t_first_token":348356.779959511,"t_first_token_unix":1779779221.7939029,"t_done":348357.755926488,"t_done_unix":1779779222.7698698},{"request_id":"1251828:1:1251828:35","session_id":"1251828","input_length":1090,"t_proxy_recv":348358.030383839,"t_decision_unix":1779779223.044325,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":1090,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":103,"cache_hit":0,"new_prefill":1090,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":96,"cache_hit":0,"new_prefill":1090,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":161,"cache_hit":0,"new_prefill":1090,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":90,"cache_hit":0,"new_prefill":1090,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":51,"cache_hit":0,"new_prefill":1090,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":13,"cache_hit":0,"new_prefill":1090,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":20086,"ongoing_decode_tokens":20086,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":105,"cache_hit":0,"new_prefill":1090,"score_linear":20086.0,"score_lmetric":1090},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":13780,"ongoing_decode_tokens":0,"pending_prefill_tokens":7636,"num_requests":1,"active_p_offloads":0,"cached_blocks":12,"cache_hit":0,"new_prefill":1090,"score_linear":13780.0,"score_lmetric":8726}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348358.121067462,"t_first_token_unix":1779779223.1350114,"t_done":348358.229893996,"t_done_unix":1779779223.2438388},{"request_id":"1250935:1:1250935:31","session_id":"1250935","input_length":20086,"t_proxy_recv":348354.970559454,"t_decision_unix":1779779219.9845006,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":20086,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":93,"cache_hit":0,"new_prefill":20086,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":96,"cache_hit":0,"new_prefill":20086,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":129,"cache_hit":0,"new_prefill":20086,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":90,"cache_hit":0,"new_prefill":20086,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":51,"cache_hit":0,"new_prefill":20086,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":6829,"ongoing_decode_tokens":0,"pending_prefill_tokens":6829,"num_requests":1,"active_p_offloads":0,"cached_blocks":0,"cache_hit":0,"new_prefill":20086,"score_linear":6829.0,"score_lmetric":26915},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":105,"cache_hit":0,"new_prefill":20086,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":12,"cache_hit":0,"new_prefill":20086,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348357.057924608,"t_first_token_unix":1779779222.0718687,"t_done":348358.316164168,"t_done_unix":1779779223.3301072},{"request_id":"1245572:3:1251662:34","session_id":"1245572","input_length":13780,"t_proxy_recv":348357.511232859,"t_decision_unix":1779779222.5251737,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":6144,"affinity_cache_ratio":0.4458635703918723,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":6144,"estimated_new_tokens":7636,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":16989,"ongoing_decode_tokens":16989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":93,"cache_hit":0,"new_prefill":13780,"score_linear":16989.0,"score_lmetric":13780},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":96,"cache_hit":0,"new_prefill":13780,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":161,"cache_hit":0,"new_prefill":13780,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":90,"cache_hit":0,"new_prefill":13780,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":51,"cache_hit":0,"new_prefill":13780,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":13,"cache_hit":0,"new_prefill":13780,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":20086,"ongoing_decode_tokens":20086,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":105,"cache_hit":0,"new_prefill":13780,"score_linear":20086.0,"score_lmetric":13780},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":12,"cache_hit":6144,"new_prefill":7636,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-6144.0,"chosen_score_lmetric":0,"t_first_token":348358.305396783,"t_first_token_unix":1779779223.31934,"t_done":348358.53384437,"t_done_unix":1779779223.5477877},{"request_id":"1252010:1:1252010:36","session_id":"1252010","input_length":9855,"t_proxy_recv":348358.639485067,"t_decision_unix":1779779223.6534264,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":9855,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":103,"cache_hit":0,"new_prefill":9855,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":96,"cache_hit":0,"new_prefill":9855,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":161,"cache_hit":0,"new_prefill":9855,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":92,"cache_hit":0,"new_prefill":9855,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":51,"cache_hit":0,"new_prefill":9855,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":13,"cache_hit":0,"new_prefill":9855,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":144,"cache_hit":0,"new_prefill":9855,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":9855,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348359.456615759,"t_first_token_unix":1779779224.4705598,"t_done":348359.627882951,"t_done_unix":1779779224.6418264},{"request_id":"1248768:4:1253385:37","session_id":"1248768","input_length":17309,"t_proxy_recv":348363.413665804,"t_decision_unix":1779779228.4276073,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":16896,"affinity_cache_ratio":0.9761395805650239,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":16896,"estimated_new_tokens":413,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":103,"cache_hit":16896,"new_prefill":413,"score_linear":-16896.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":96,"cache_hit":0,"new_prefill":17309,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":161,"cache_hit":0,"new_prefill":17309,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":92,"cache_hit":0,"new_prefill":17309,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":51,"cache_hit":0,"new_prefill":17309,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":13,"cache_hit":0,"new_prefill":17309,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":17309,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":17309,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-16896.0,"chosen_score_lmetric":0,"t_first_token":348363.499118061,"t_first_token_unix":1779779228.513061,"t_done":348366.472771927,"t_done_unix":1779779231.4867148},{"request_id":"1251828:2:1254272:41","session_id":"1251828","input_length":3934,"t_proxy_recv":348366.382243649,"t_decision_unix":1779779231.3961852,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":3,"chosen_idx":4,"affinity_cache_hit":1024,"affinity_cache_ratio":0.2602948652770717,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":3934,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":17309,"ongoing_decode_tokens":17309,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":103,"cache_hit":0,"new_prefill":3934,"score_linear":17309.0,"score_lmetric":3934},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":35221,"ongoing_decode_tokens":0,"pending_prefill_tokens":35221,"num_requests":1,"active_p_offloads":0,"cached_blocks":96,"cache_hit":0,"new_prefill":3934,"score_linear":35221.0,"score_lmetric":39155},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":161,"cache_hit":0,"new_prefill":3934,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":49624,"ongoing_decode_tokens":0,"pending_prefill_tokens":49624,"num_requests":1,"active_p_offloads":0,"cached_blocks":92,"cache_hit":1024,"new_prefill":2910,"score_linear":48600.0,"score_lmetric":52534},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":51,"cache_hit":0,"new_prefill":3934,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":10764,"ongoing_decode_tokens":0,"pending_prefill_tokens":4108,"num_requests":1,"active_p_offloads":0,"cached_blocks":13,"cache_hit":0,"new_prefill":3934,"score_linear":10764.0,"score_lmetric":8042},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":3934,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":3934,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348366.62874853,"t_first_token_unix":1779779231.6426916,"t_done":348366.774855527,"t_done_unix":1779779231.7887986},{"request_id":"1254584:1:1254584:42","session_id":"1254584","input_length":748,"t_proxy_recv":348367.515368298,"t_decision_unix":1779779232.5293097,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":748,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":104,"cache_hit":0,"new_prefill":748,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":35221,"ongoing_decode_tokens":0,"pending_prefill_tokens":35221,"num_requests":1,"active_p_offloads":0,"cached_blocks":96,"cache_hit":0,"new_prefill":748,"score_linear":35221.0,"score_lmetric":35969},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":161,"cache_hit":0,"new_prefill":748,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":49624,"ongoing_decode_tokens":0,"pending_prefill_tokens":49624,"num_requests":1,"active_p_offloads":0,"cached_blocks":92,"cache_hit":0,"new_prefill":748,"score_linear":49624.0,"score_lmetric":50372},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":58,"cache_hit":0,"new_prefill":748,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":10764,"ongoing_decode_tokens":10764,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":13,"cache_hit":0,"new_prefill":748,"score_linear":10764.0,"score_lmetric":748},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":748,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":748,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348367.584711415,"t_first_token_unix":1779779232.5986545,"t_done":348367.738787482,"t_done_unix":1779779232.7527304},{"request_id":"1252010:2:1254691:43","session_id":"1252010","input_length":13092,"t_proxy_recv":348367.968085608,"t_decision_unix":1779779232.9820273,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":9728,"affinity_cache_ratio":0.743049190345249,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":9728,"estimated_new_tokens":3364,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":104,"cache_hit":0,"new_prefill":13092,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":35221,"ongoing_decode_tokens":0,"pending_prefill_tokens":35221,"num_requests":1,"active_p_offloads":0,"cached_blocks":96,"cache_hit":0,"new_prefill":13092,"score_linear":35221.0,"score_lmetric":48313},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":162,"cache_hit":0,"new_prefill":13092,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":49624,"ongoing_decode_tokens":0,"pending_prefill_tokens":49624,"num_requests":1,"active_p_offloads":0,"cached_blocks":92,"cache_hit":0,"new_prefill":13092,"score_linear":49624.0,"score_lmetric":62716},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":58,"cache_hit":0,"new_prefill":13092,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":10764,"ongoing_decode_tokens":10764,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":13,"cache_hit":0,"new_prefill":13092,"score_linear":10764.0,"score_lmetric":13092},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":163,"cache_hit":9728,"new_prefill":3364,"score_linear":-9728.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":13092,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-9728.0,"chosen_score_lmetric":0,"t_first_token":348368.395540758,"t_first_token_unix":1779779233.4094841,"t_done":348368.564723316,"t_done_unix":1779779233.5786662},{"request_id":"1250772:2:1254198:40","session_id":"1250772","input_length":10764,"t_proxy_recv":348366.109711656,"t_decision_unix":1779779231.1236532,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":6656,"affinity_cache_ratio":0.6183574879227053,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":6656,"estimated_new_tokens":4108,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":17309,"ongoing_decode_tokens":17309,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":103,"cache_hit":0,"new_prefill":10764,"score_linear":17309.0,"score_lmetric":10764},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":35221,"ongoing_decode_tokens":0,"pending_prefill_tokens":35221,"num_requests":1,"active_p_offloads":0,"cached_blocks":96,"cache_hit":0,"new_prefill":10764,"score_linear":35221.0,"score_lmetric":45985},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":161,"cache_hit":0,"new_prefill":10764,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":49624,"ongoing_decode_tokens":0,"pending_prefill_tokens":49624,"num_requests":1,"active_p_offloads":0,"cached_blocks":92,"cache_hit":0,"new_prefill":10764,"score_linear":49624.0,"score_lmetric":60388},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":51,"cache_hit":0,"new_prefill":10764,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":13,"cache_hit":6656,"new_prefill":4108,"score_linear":-6656.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":10764,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":10764,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-6656.0,"chosen_score_lmetric":0,"t_first_token":348366.493210954,"t_first_token_unix":1779779231.5071542,"t_done":348368.766580387,"t_done_unix":1779779233.7805235},{"request_id":"1253804:1:1253804:39","session_id":"1253804","input_length":35221,"t_proxy_recv":348365.003287018,"t_decision_unix":1779779230.0172286,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":35221,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":17309,"ongoing_decode_tokens":17309,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":103,"cache_hit":0,"new_prefill":35221,"score_linear":17309.0,"score_lmetric":35221},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":96,"cache_hit":0,"new_prefill":35221,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":161,"cache_hit":0,"new_prefill":35221,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":49624,"ongoing_decode_tokens":0,"pending_prefill_tokens":49624,"num_requests":1,"active_p_offloads":0,"cached_blocks":92,"cache_hit":0,"new_prefill":35221,"score_linear":49624.0,"score_lmetric":84845},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":51,"cache_hit":0,"new_prefill":35221,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":13,"cache_hit":0,"new_prefill":35221,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":35221,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":35221,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348370.16412528,"t_first_token_unix":1779779235.1780686,"t_done":348372.386767828,"t_done_unix":1779779237.4007115},{"request_id":"1256273:1:1256273:46","session_id":"1256273","input_length":1694,"t_proxy_recv":348373.197457375,"t_decision_unix":1779779238.2113988,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":1694,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":104,"cache_hit":0,"new_prefill":1694,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":165,"cache_hit":0,"new_prefill":1694,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":162,"cache_hit":0,"new_prefill":1694,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":49624,"ongoing_decode_tokens":0,"pending_prefill_tokens":49624,"num_requests":1,"active_p_offloads":0,"cached_blocks":92,"cache_hit":0,"new_prefill":1694,"score_linear":49624.0,"score_lmetric":51318},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":17786,"ongoing_decode_tokens":0,"pending_prefill_tokens":10618,"num_requests":1,"active_p_offloads":0,"cached_blocks":58,"cache_hit":0,"new_prefill":1694,"score_linear":17786.0,"score_lmetric":12312},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":21,"cache_hit":0,"new_prefill":1694,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":169,"cache_hit":0,"new_prefill":1694,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":1694,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348373.348118357,"t_first_token_unix":1779779238.3620613,"t_done":348373.633032233,"t_done_unix":1779779238.6469753},{"request_id":"1243831:2:1256324:47","session_id":"1243831","input_length":11448,"t_proxy_recv":348373.362542591,"t_decision_unix":1779779238.3764842,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":11264,"affinity_cache_ratio":0.983927323549965,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":11264,"estimated_new_tokens":184,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":104,"cache_hit":0,"new_prefill":11448,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":165,"cache_hit":11264,"new_prefill":184,"score_linear":-11264.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":162,"cache_hit":0,"new_prefill":11448,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":49624,"ongoing_decode_tokens":0,"pending_prefill_tokens":49624,"num_requests":1,"active_p_offloads":0,"cached_blocks":92,"cache_hit":0,"new_prefill":11448,"score_linear":49624.0,"score_lmetric":61072},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":17786,"ongoing_decode_tokens":0,"pending_prefill_tokens":10618,"num_requests":1,"active_p_offloads":0,"cached_blocks":58,"cache_hit":0,"new_prefill":11448,"score_linear":17786.0,"score_lmetric":22066},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":1694,"ongoing_decode_tokens":1694,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":21,"cache_hit":0,"new_prefill":11448,"score_linear":1694.0,"score_lmetric":11448},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":169,"cache_hit":0,"new_prefill":11448,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":11448,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-11264.0,"chosen_score_lmetric":0,"t_first_token":348373.415353514,"t_first_token_unix":1779779238.4292967,"t_done":348373.856157566,"t_done_unix":1779779238.8701007},{"request_id":"1253743:1:1253743:38","session_id":"1253743","input_length":49624,"t_proxy_recv":348364.757886534,"t_decision_unix":1779779229.771828,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":49624,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":17309,"ongoing_decode_tokens":17309,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":103,"cache_hit":0,"new_prefill":49624,"score_linear":17309.0,"score_lmetric":49624},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":96,"cache_hit":0,"new_prefill":49624,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":161,"cache_hit":0,"new_prefill":49624,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":92,"cache_hit":0,"new_prefill":49624,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":51,"cache_hit":0,"new_prefill":49624,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":13,"cache_hit":0,"new_prefill":49624,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":49624,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":49624,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348373.99738589,"t_first_token_unix":1779779239.0113297,"t_done":348374.400073102,"t_done_unix":1779779239.4140162},{"request_id":"1249058:3:1256222:45","session_id":"1249058","input_length":17786,"t_proxy_recv":348373.051055498,"t_decision_unix":1779779238.0649967,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":7168,"affinity_cache_ratio":0.4030136062071292,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":7168,"estimated_new_tokens":10618,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":104,"cache_hit":0,"new_prefill":17786,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":165,"cache_hit":0,"new_prefill":17786,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":162,"cache_hit":0,"new_prefill":17786,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":49624,"ongoing_decode_tokens":0,"pending_prefill_tokens":49624,"num_requests":1,"active_p_offloads":0,"cached_blocks":92,"cache_hit":0,"new_prefill":17786,"score_linear":49624.0,"score_lmetric":67410},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":58,"cache_hit":7168,"new_prefill":10618,"score_linear":-7168.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":21,"cache_hit":0,"new_prefill":17786,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":169,"cache_hit":0,"new_prefill":17786,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":17786,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-7168.0,"chosen_score_lmetric":0,"t_first_token":348374.317068255,"t_first_token_unix":1779779239.3310118,"t_done":348374.435212041,"t_done_unix":1779779239.4491549},{"request_id":"1253743:2:1256070:44","session_id":"1253743","input_length":49687,"t_proxy_recv":348374.409966961,"t_decision_unix":1779779239.4239087,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":49664,"affinity_cache_ratio":0.9995371022601486,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":49664,"estimated_new_tokens":23,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":104,"cache_hit":0,"new_prefill":49687,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":165,"cache_hit":3584,"new_prefill":46103,"score_linear":-3584.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":162,"cache_hit":0,"new_prefill":49687,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":189,"cache_hit":49664,"new_prefill":23,"score_linear":-49664.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":17786,"ongoing_decode_tokens":17786,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":58,"cache_hit":0,"new_prefill":49687,"score_linear":17786.0,"score_lmetric":49687},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":24,"cache_hit":0,"new_prefill":49687,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":169,"cache_hit":0,"new_prefill":49687,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":49687,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-49664.0,"chosen_score_lmetric":0,"t_first_token":348374.494898377,"t_first_token_unix":1779779239.5088415,"t_done":348375.319770357,"t_done_unix":1779779240.3337138},{"request_id":"1256273:2:1257769:51","session_id":"1256273","input_length":6170,"t_proxy_recv":348378.181679317,"t_decision_unix":1779779243.1956205,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":1536,"affinity_cache_ratio":0.24894651539708265,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":1536,"estimated_new_tokens":4634,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":4434,"ongoing_decode_tokens":4434,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":104,"cache_hit":0,"new_prefill":6170,"score_linear":4434.0,"score_lmetric":6170},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":27387,"ongoing_decode_tokens":0,"pending_prefill_tokens":27387,"num_requests":1,"active_p_offloads":0,"cached_blocks":165,"cache_hit":0,"new_prefill":6170,"score_linear":27387.0,"score_lmetric":33557},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":162,"cache_hit":0,"new_prefill":6170,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":189,"cache_hit":0,"new_prefill":6170,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90488,"ongoing_decode_tokens":0,"pending_prefill_tokens":90488,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":6170,"score_linear":90488.0,"score_lmetric":96658},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":24,"cache_hit":1536,"new_prefill":4634,"score_linear":-1536.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":169,"cache_hit":0,"new_prefill":6170,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":6170,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-1536.0,"chosen_score_lmetric":0,"t_first_token":348378.507053323,"t_first_token_unix":1779779243.5209966,"t_done":348378.812334098,"t_done_unix":1779779243.826277},{"request_id":"1257572:1:1257572:50","session_id":"1257572","input_length":4434,"t_proxy_recv":348377.546548874,"t_decision_unix":1779779242.5604906,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":4434,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":104,"cache_hit":0,"new_prefill":4434,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":27387,"ongoing_decode_tokens":0,"pending_prefill_tokens":27387,"num_requests":1,"active_p_offloads":0,"cached_blocks":165,"cache_hit":0,"new_prefill":4434,"score_linear":27387.0,"score_lmetric":31821},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":162,"cache_hit":0,"new_prefill":4434,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":189,"cache_hit":0,"new_prefill":4434,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90488,"ongoing_decode_tokens":0,"pending_prefill_tokens":90488,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":4434,"score_linear":90488.0,"score_lmetric":94922},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":24,"cache_hit":0,"new_prefill":4434,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":169,"cache_hit":0,"new_prefill":4434,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":4434,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348377.832896271,"t_first_token_unix":1779779242.8468397,"t_done":348379.07587086,"t_done_unix":1779779244.089814},{"request_id":"1254584:2:1258039:52","session_id":"1254584","input_length":1257,"t_proxy_recv":348379.201232143,"t_decision_unix":1779779244.2151737,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":512,"affinity_cache_ratio":0.40731901352426414,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":512,"estimated_new_tokens":745,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":113,"cache_hit":0,"new_prefill":1257,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":27387,"ongoing_decode_tokens":0,"pending_prefill_tokens":27387,"num_requests":1,"active_p_offloads":0,"cached_blocks":165,"cache_hit":0,"new_prefill":1257,"score_linear":27387.0,"score_lmetric":28644},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":162,"cache_hit":512,"new_prefill":745,"score_linear":-512.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":189,"cache_hit":0,"new_prefill":1257,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90488,"ongoing_decode_tokens":0,"pending_prefill_tokens":90488,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":1257,"score_linear":90488.0,"score_lmetric":91745},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":33,"cache_hit":0,"new_prefill":1257,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":169,"cache_hit":0,"new_prefill":1257,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":1257,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-512.0,"chosen_score_lmetric":0,"t_first_token":348379.255663116,"t_first_token_unix":1779779244.269606,"t_done":348379.591043299,"t_done_unix":1779779244.6049864},{"request_id":"1242838:3:1258161:53","session_id":"1242838","input_length":22262,"t_proxy_recv":348379.661635233,"t_decision_unix":1779779244.675577,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":21504,"affinity_cache_ratio":0.9659509478034318,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":21504,"estimated_new_tokens":758,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":113,"cache_hit":21504,"new_prefill":758,"score_linear":-21504.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":27387,"ongoing_decode_tokens":0,"pending_prefill_tokens":27387,"num_requests":1,"active_p_offloads":0,"cached_blocks":165,"cache_hit":0,"new_prefill":22262,"score_linear":27387.0,"score_lmetric":49649},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":22262,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":189,"cache_hit":0,"new_prefill":22262,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90488,"ongoing_decode_tokens":0,"pending_prefill_tokens":90488,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":22262,"score_linear":90488.0,"score_lmetric":112750},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":33,"cache_hit":0,"new_prefill":22262,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":169,"cache_hit":0,"new_prefill":22262,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":22262,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-21504.0,"chosen_score_lmetric":0,"t_first_token":348379.837060632,"t_first_token_unix":1779779244.851004,"t_done":348380.378039934,"t_done_unix":1779779245.3919828},{"request_id":"1256273:3:1258499:55","session_id":"1256273","input_length":7934,"t_proxy_recv":348380.825506707,"t_decision_unix":1779779245.839448,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":6144,"affinity_cache_ratio":0.7743887068313587,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":6144,"estimated_new_tokens":1790,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":114,"cache_hit":0,"new_prefill":7934,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":80365,"ongoing_decode_tokens":0,"pending_prefill_tokens":45037,"num_requests":2,"active_p_offloads":0,"cached_blocks":165,"cache_hit":0,"new_prefill":7934,"score_linear":80365.0,"score_lmetric":105942},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":7934,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":189,"cache_hit":0,"new_prefill":7934,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90488,"ongoing_decode_tokens":0,"pending_prefill_tokens":90488,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":7934,"score_linear":90488.0,"score_lmetric":98422},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":33,"cache_hit":6144,"new_prefill":1790,"score_linear":-6144.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":169,"cache_hit":0,"new_prefill":7934,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":7934,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-6144.0,"chosen_score_lmetric":0,"t_first_token":348381.010957161,"t_first_token_unix":1779779246.0249002,"t_done":348381.628742384,"t_done_unix":1779779246.6426854},{"request_id":"1258908:1:1258908:58","session_id":"1258908","input_length":9250,"t_proxy_recv":348382.278119848,"t_decision_unix":1779779247.2920613,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":9250,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":114,"cache_hit":0,"new_prefill":9250,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":80365,"ongoing_decode_tokens":27387,"pending_prefill_tokens":17650,"num_requests":2,"active_p_offloads":0,"cached_blocks":165,"cache_hit":0,"new_prefill":9250,"score_linear":80365.0,"score_lmetric":53800},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":68131,"ongoing_decode_tokens":0,"pending_prefill_tokens":68131,"num_requests":1,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":9250,"score_linear":68131.0,"score_lmetric":77381},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":189,"cache_hit":0,"new_prefill":9250,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90488,"ongoing_decode_tokens":0,"pending_prefill_tokens":90488,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":9250,"score_linear":90488.0,"score_lmetric":99738},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":112793,"ongoing_decode_tokens":0,"pending_prefill_tokens":112793,"num_requests":1,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":9250,"score_linear":112793.0,"score_lmetric":122043},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":169,"cache_hit":0,"new_prefill":9250,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":9250,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348382.974789085,"t_first_token_unix":1779779247.9887326,"t_done":348384.977667664,"t_done_unix":1779779249.9916112},{"request_id":"1259178:1:1259178:59","session_id":"1259178","input_length":14291,"t_proxy_recv":348383.240075013,"t_decision_unix":1779779248.2540164,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":14291,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":114,"cache_hit":0,"new_prefill":14291,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":80365,"ongoing_decode_tokens":27387,"pending_prefill_tokens":17650,"num_requests":2,"active_p_offloads":0,"cached_blocks":165,"cache_hit":0,"new_prefill":14291,"score_linear":80365.0,"score_lmetric":63882},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":68131,"ongoing_decode_tokens":0,"pending_prefill_tokens":68131,"num_requests":1,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":14291,"score_linear":68131.0,"score_lmetric":82422},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":9250,"ongoing_decode_tokens":9250,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":189,"cache_hit":0,"new_prefill":14291,"score_linear":9250.0,"score_lmetric":14291},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90488,"ongoing_decode_tokens":0,"pending_prefill_tokens":90488,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":14291,"score_linear":90488.0,"score_lmetric":104779},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":112793,"ongoing_decode_tokens":0,"pending_prefill_tokens":112793,"num_requests":1,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":14291,"score_linear":112793.0,"score_lmetric":127084},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":169,"cache_hit":0,"new_prefill":14291,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":14291,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348384.494019127,"t_first_token_unix":1779779249.5079632,"t_done":348386.443920618,"t_done_unix":1779779251.4578643},{"request_id":"1257525:1:1257525:49","session_id":"1257525","input_length":27387,"t_proxy_recv":348377.46107748,"t_decision_unix":1779779242.4750187,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":27387,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":104,"cache_hit":0,"new_prefill":27387,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":165,"cache_hit":0,"new_prefill":27387,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":162,"cache_hit":0,"new_prefill":27387,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":189,"cache_hit":0,"new_prefill":27387,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90488,"ongoing_decode_tokens":0,"pending_prefill_tokens":90488,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":27387,"score_linear":90488.0,"score_lmetric":117875},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":24,"cache_hit":0,"new_prefill":27387,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":169,"cache_hit":0,"new_prefill":27387,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":27387,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348380.91020456,"t_first_token_unix":1779779245.9241476,"t_done":348386.538905236,"t_done_unix":1779779251.5528483},{"request_id":"1253804:2:1258346:54","session_id":"1253804","input_length":52978,"t_proxy_recv":348380.372254888,"t_decision_unix":1779779245.3861961,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":35328,"affinity_cache_ratio":0.6668428404243271,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":35328,"estimated_new_tokens":17650,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":22262,"ongoing_decode_tokens":22262,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":113,"cache_hit":0,"new_prefill":52978,"score_linear":22262.0,"score_lmetric":52978},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":27387,"ongoing_decode_tokens":0,"pending_prefill_tokens":27387,"num_requests":1,"active_p_offloads":0,"cached_blocks":165,"cache_hit":35328,"new_prefill":17650,"score_linear":-7941.0,"score_lmetric":45037},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":52978,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":189,"cache_hit":3584,"new_prefill":49394,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90488,"ongoing_decode_tokens":0,"pending_prefill_tokens":90488,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":52978,"score_linear":90488.0,"score_lmetric":143466},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":33,"cache_hit":0,"new_prefill":52978,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":169,"cache_hit":0,"new_prefill":52978,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":52978,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-7941.0,"chosen_score_lmetric":45037,"t_first_token":348386.161918926,"t_first_token_unix":1779779251.1758628,"t_done":348387.092948804,"t_done_unix":1779779252.1068919},{"request_id":"1260746:1:1260746:62","session_id":"1260746","input_length":9571,"t_proxy_recv":348388.784299041,"t_decision_unix":1779779253.79824,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":9571,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":114,"cache_hit":0,"new_prefill":9571,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":252,"cache_hit":0,"new_prefill":9571,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":68131,"ongoing_decode_tokens":0,"pending_prefill_tokens":68131,"num_requests":1,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":9571,"score_linear":68131.0,"score_lmetric":77702},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":9571,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90488,"ongoing_decode_tokens":0,"pending_prefill_tokens":90488,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":9571,"score_linear":90488.0,"score_lmetric":100059},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":9571,"score_linear":125063.0,"score_lmetric":253908},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":197,"cache_hit":0,"new_prefill":9571,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":30655,"ongoing_decode_tokens":0,"pending_prefill_tokens":30655,"num_requests":1,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":9571,"score_linear":30655.0,"score_lmetric":40226}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348389.541674131,"t_first_token_unix":1779779254.555618,"t_done":348389.933484861,"t_done_unix":1779779254.9474285},{"request_id":"1260792:1:1260792:63","session_id":"1260792","input_length":12522,"t_proxy_recv":348388.900132629,"t_decision_unix":1779779253.9140737,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":12522,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":9571,"ongoing_decode_tokens":0,"pending_prefill_tokens":9571,"num_requests":1,"active_p_offloads":0,"cached_blocks":114,"cache_hit":0,"new_prefill":12522,"score_linear":9571.0,"score_lmetric":22093},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":252,"cache_hit":0,"new_prefill":12522,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":68131,"ongoing_decode_tokens":0,"pending_prefill_tokens":68131,"num_requests":1,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":12522,"score_linear":68131.0,"score_lmetric":80653},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":12522,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90488,"ongoing_decode_tokens":0,"pending_prefill_tokens":90488,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":12522,"score_linear":90488.0,"score_lmetric":103010},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":12522,"score_linear":125063.0,"score_lmetric":259810},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":197,"cache_hit":0,"new_prefill":12522,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":30655,"ongoing_decode_tokens":0,"pending_prefill_tokens":30655,"num_requests":1,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":12522,"score_linear":30655.0,"score_lmetric":43177}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348389.93395426,"t_first_token_unix":1779779254.9478977,"t_done":348390.265884821,"t_done_unix":1779779255.2798283},{"request_id":"1261126:1:1261126:65","session_id":"1261126","input_length":5640,"t_proxy_recv":348389.937952171,"t_decision_unix":1779779254.9518938,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":5640,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":132,"cache_hit":0,"new_prefill":5640,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55138,"ongoing_decode_tokens":0,"pending_prefill_tokens":2402,"num_requests":1,"active_p_offloads":0,"cached_blocks":252,"cache_hit":0,"new_prefill":5640,"score_linear":55138.0,"score_lmetric":8042},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":68131,"ongoing_decode_tokens":0,"pending_prefill_tokens":68131,"num_requests":1,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":5640,"score_linear":68131.0,"score_lmetric":73771},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":12522,"ongoing_decode_tokens":12522,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":5640,"score_linear":12522.0,"score_lmetric":5640},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90488,"ongoing_decode_tokens":0,"pending_prefill_tokens":90488,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":5640,"score_linear":90488.0,"score_lmetric":96128},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":5640,"score_linear":125063.0,"score_lmetric":246046},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":197,"cache_hit":0,"new_prefill":5640,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":30655,"ongoing_decode_tokens":0,"pending_prefill_tokens":30655,"num_requests":1,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":5640,"score_linear":30655.0,"score_lmetric":36295}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348390.307456923,"t_first_token_unix":1779779255.3214006,"t_done":348390.476180727,"t_done_unix":1779779255.4901237},{"request_id":"1253804:3:1260997:64","session_id":"1253804","input_length":55138,"t_proxy_recv":348389.639395733,"t_decision_unix":1779779254.653337,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":52736,"affinity_cache_ratio":0.9564365773151003,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":52736,"estimated_new_tokens":2402,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":9571,"ongoing_decode_tokens":9571,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":114,"cache_hit":0,"new_prefill":55138,"score_linear":9571.0,"score_lmetric":55138},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":252,"cache_hit":52736,"new_prefill":2402,"score_linear":-52736.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":68131,"ongoing_decode_tokens":0,"pending_prefill_tokens":68131,"num_requests":1,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":55138,"score_linear":68131.0,"score_lmetric":123269},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":12522,"ongoing_decode_tokens":0,"pending_prefill_tokens":12522,"num_requests":1,"active_p_offloads":0,"cached_blocks":207,"cache_hit":3584,"new_prefill":51554,"score_linear":8938.0,"score_lmetric":64076},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90488,"ongoing_decode_tokens":0,"pending_prefill_tokens":90488,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":55138,"score_linear":90488.0,"score_lmetric":145626},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":55138,"score_linear":125063.0,"score_lmetric":345042},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":197,"cache_hit":0,"new_prefill":55138,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":30655,"ongoing_decode_tokens":0,"pending_prefill_tokens":30655,"num_requests":1,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":55138,"score_linear":30655.0,"score_lmetric":85793}],"chosen_score_linear":-52736.0,"chosen_score_lmetric":0,"t_first_token":348390.508104581,"t_first_token_unix":1779779255.5220482,"t_done":348391.103970418,"t_done_unix":1779779256.1179144},{"request_id":"1260327:1:1260327:61","session_id":"1260327","input_length":30655,"t_proxy_recv":348387.064984977,"t_decision_unix":1779779252.0789263,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":30655,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":114,"cache_hit":0,"new_prefill":30655,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":52978,"ongoing_decode_tokens":52978,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":218,"cache_hit":0,"new_prefill":30655,"score_linear":52978.0,"score_lmetric":30655},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":68131,"ongoing_decode_tokens":0,"pending_prefill_tokens":68131,"num_requests":1,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":30655,"score_linear":68131.0,"score_lmetric":98786},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":30655,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90488,"ongoing_decode_tokens":0,"pending_prefill_tokens":90488,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":30655,"score_linear":90488.0,"score_lmetric":121143},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":30655,"score_linear":125063.0,"score_lmetric":296076},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":197,"cache_hit":0,"new_prefill":30655,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":30655,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348391.113601316,"t_first_token_unix":1779779256.1275449,"t_done":348391.767262777,"t_done_unix":1779779256.7812068},{"request_id":"1257525:2:1262205:67","session_id":"1257525","input_length":29988,"t_proxy_recv":348393.753531824,"t_decision_unix":1779779258.767473,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":27136,"affinity_cache_ratio":0.9048952914499133,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":27136,"estimated_new_tokens":2852,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":143,"cache_hit":0,"new_prefill":29988,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":256,"cache_hit":27136,"new_prefill":2852,"score_linear":-27136.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":68131,"ongoing_decode_tokens":0,"pending_prefill_tokens":68131,"num_requests":1,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":29988,"score_linear":68131.0,"score_lmetric":98119},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":29988,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90488,"ongoing_decode_tokens":0,"pending_prefill_tokens":90488,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":29988,"score_linear":90488.0,"score_lmetric":120476},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":29988,"score_linear":125063.0,"score_lmetric":294742},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":197,"cache_hit":0,"new_prefill":29988,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":86,"cache_hit":0,"new_prefill":29988,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-27136.0,"chosen_score_lmetric":0,"t_first_token":348394.37446006,"t_first_token_unix":1779779259.3884034,"t_done":348394.407380861,"t_done_unix":1779779259.4213238},{"request_id":"1262757:1:1262757:73","session_id":"1262757","input_length":5465,"t_proxy_recv":348395.633082392,"t_decision_unix":1779779260.6470242,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":5465,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":143,"cache_hit":0,"new_prefill":5465,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55989,"ongoing_decode_tokens":0,"pending_prefill_tokens":1205,"num_requests":1,"active_p_offloads":0,"cached_blocks":261,"cache_hit":0,"new_prefill":5465,"score_linear":55989.0,"score_lmetric":6670},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":68131,"ongoing_decode_tokens":0,"pending_prefill_tokens":68131,"num_requests":1,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":5465,"score_linear":68131.0,"score_lmetric":73596},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":0,"pending_prefill_tokens":15283,"num_requests":1,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":5465,"score_linear":15283.0,"score_lmetric":20748},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":113053,"ongoing_decode_tokens":0,"pending_prefill_tokens":92573,"num_requests":2,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":5465,"score_linear":113053.0,"score_lmetric":196076},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":5465,"score_linear":125063.0,"score_lmetric":245696},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":197,"cache_hit":0,"new_prefill":5465,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":21510,"ongoing_decode_tokens":0,"pending_prefill_tokens":21510,"num_requests":1,"active_p_offloads":0,"cached_blocks":86,"cache_hit":0,"new_prefill":5465,"score_linear":21510.0,"score_lmetric":26975}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348395.996692977,"t_first_token_unix":1779779261.0106363,"t_done":348396.127976956,"t_done_unix":1779779261.14192},{"request_id":"1261126:2:1262820:74","session_id":"1261126","input_length":6953,"t_proxy_recv":348395.805630463,"t_decision_unix":1779779260.8195715,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":5632,"affinity_cache_ratio":0.8100100675967209,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":5632,"estimated_new_tokens":1321,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":143,"cache_hit":5632,"new_prefill":1321,"score_linear":-5632.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55989,"ongoing_decode_tokens":0,"pending_prefill_tokens":1205,"num_requests":1,"active_p_offloads":0,"cached_blocks":261,"cache_hit":0,"new_prefill":6953,"score_linear":55989.0,"score_lmetric":8158},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":68131,"ongoing_decode_tokens":0,"pending_prefill_tokens":68131,"num_requests":1,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":6953,"score_linear":68131.0,"score_lmetric":75084},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":0,"pending_prefill_tokens":15283,"num_requests":1,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":6953,"score_linear":15283.0,"score_lmetric":22236},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":113053,"ongoing_decode_tokens":0,"pending_prefill_tokens":92573,"num_requests":2,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":6953,"score_linear":113053.0,"score_lmetric":199052},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":6953,"score_linear":125063.0,"score_lmetric":248672},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":5465,"ongoing_decode_tokens":0,"pending_prefill_tokens":5465,"num_requests":1,"active_p_offloads":0,"cached_blocks":197,"cache_hit":0,"new_prefill":6953,"score_linear":5465.0,"score_lmetric":12418},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":21510,"ongoing_decode_tokens":0,"pending_prefill_tokens":21510,"num_requests":1,"active_p_offloads":0,"cached_blocks":86,"cache_hit":0,"new_prefill":6953,"score_linear":21510.0,"score_lmetric":28463}],"chosen_score_linear":-5632.0,"chosen_score_lmetric":0,"t_first_token":348395.961742564,"t_first_token_unix":1779779260.9756858,"t_done":348396.199076107,"t_done_unix":1779779261.213019},{"request_id":"1262354:1:1262354:69","session_id":"1262354","input_length":21510,"t_proxy_recv":348394.349613607,"t_decision_unix":1779779259.363555,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":21510,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":143,"cache_hit":0,"new_prefill":21510,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":29988,"ongoing_decode_tokens":0,"pending_prefill_tokens":2852,"num_requests":1,"active_p_offloads":0,"cached_blocks":256,"cache_hit":0,"new_prefill":21510,"score_linear":29988.0,"score_lmetric":24362},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":68131,"ongoing_decode_tokens":0,"pending_prefill_tokens":68131,"num_requests":1,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":21510,"score_linear":68131.0,"score_lmetric":89641},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":21510,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":113053,"ongoing_decode_tokens":0,"pending_prefill_tokens":92573,"num_requests":2,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":21510,"score_linear":113053.0,"score_lmetric":228166},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":21510,"score_linear":125063.0,"score_lmetric":277786},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":197,"cache_hit":0,"new_prefill":21510,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":86,"cache_hit":0,"new_prefill":21510,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348396.652738613,"t_first_token_unix":1779779261.6666818,"t_done":348397.260913409,"t_done_unix":1779779262.274857},{"request_id":"1258611:1:1258611:56","session_id":"1258611","input_length":68131,"t_proxy_recv":348381.224270921,"t_decision_unix":1779779246.2382123,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":68131,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":114,"cache_hit":0,"new_prefill":68131,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":80365,"ongoing_decode_tokens":27387,"pending_prefill_tokens":17650,"num_requests":2,"active_p_offloads":0,"cached_blocks":165,"cache_hit":0,"new_prefill":68131,"score_linear":80365.0,"score_lmetric":171562},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":68131,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":189,"cache_hit":0,"new_prefill":68131,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90488,"ongoing_decode_tokens":0,"pending_prefill_tokens":90488,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":68131,"score_linear":90488.0,"score_lmetric":158619},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":7934,"ongoing_decode_tokens":7934,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":33,"cache_hit":0,"new_prefill":68131,"score_linear":7934.0,"score_lmetric":68131},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":169,"cache_hit":0,"new_prefill":68131,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":68131,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348397.554683891,"t_first_token_unix":1779779262.5686274,"t_done":348398.726661063,"t_done_unix":1779779263.7406042},{"request_id":"1258611:2:1261594:66","session_id":"1258611","input_length":68320,"t_proxy_recv":348398.740677272,"t_decision_unix":1779779263.754619,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":68096,"affinity_cache_ratio":0.9967213114754099,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":68096,"estimated_new_tokens":224,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":145,"cache_hit":0,"new_prefill":68320,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55989,"ongoing_decode_tokens":55989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":261,"cache_hit":0,"new_prefill":68320,"score_linear":55989.0,"score_lmetric":68320},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":296,"cache_hit":68096,"new_prefill":224,"score_linear":-68096.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":68320,"score_linear":15283.0,"score_lmetric":68320},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":113053,"ongoing_decode_tokens":0,"pending_prefill_tokens":92573,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":68320,"score_linear":113053.0,"score_lmetric":160893},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":68320,"score_linear":125063.0,"score_lmetric":371406},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":68320,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":22109,"ongoing_decode_tokens":0,"pending_prefill_tokens":605,"num_requests":1,"active_p_offloads":0,"cached_blocks":128,"cache_hit":0,"new_prefill":68320,"score_linear":22109.0,"score_lmetric":68925}],"chosen_score_linear":-68096.0,"chosen_score_lmetric":0,"t_first_token":348398.884829052,"t_first_token_unix":1779779263.8987722,"t_done":348398.921810016,"t_done_unix":1779779263.935753},{"request_id":"1258611:3:1262709:71","session_id":"1258611","input_length":68377,"t_proxy_recv":348398.932239177,"t_decision_unix":1779779263.9461808,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":68096,"affinity_cache_ratio":0.9958904309928778,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":68096,"estimated_new_tokens":281,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":5990,"ongoing_decode_tokens":0,"pending_prefill_tokens":5990,"num_requests":1,"active_p_offloads":0,"cached_blocks":145,"cache_hit":0,"new_prefill":68377,"score_linear":5990.0,"score_lmetric":74367},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55989,"ongoing_decode_tokens":55989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":261,"cache_hit":0,"new_prefill":68377,"score_linear":55989.0,"score_lmetric":68377},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":296,"cache_hit":68096,"new_prefill":281,"score_linear":-68096.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":68377,"score_linear":15283.0,"score_lmetric":68377},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":113053,"ongoing_decode_tokens":0,"pending_prefill_tokens":92573,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":68377,"score_linear":113053.0,"score_lmetric":160950},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":68377,"score_linear":125063.0,"score_lmetric":371520},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":68377,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":22109,"ongoing_decode_tokens":22109,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":128,"cache_hit":0,"new_prefill":68377,"score_linear":22109.0,"score_lmetric":68377}],"chosen_score_linear":-68096.0,"chosen_score_lmetric":0,"t_first_token":348399.06785866,"t_first_token_unix":1779779264.0818017,"t_done":348399.506565116,"t_done_unix":1779779264.520509},{"request_id":"1263611:1:1263611:76","session_id":"1263611","input_length":5990,"t_proxy_recv":348398.827430686,"t_decision_unix":1779779263.8413725,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":5990,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":145,"cache_hit":0,"new_prefill":5990,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55989,"ongoing_decode_tokens":55989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":261,"cache_hit":0,"new_prefill":5990,"score_linear":55989.0,"score_lmetric":5990},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":68320,"ongoing_decode_tokens":0,"pending_prefill_tokens":224,"num_requests":1,"active_p_offloads":0,"cached_blocks":296,"cache_hit":0,"new_prefill":5990,"score_linear":68320.0,"score_lmetric":6214},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":5990,"score_linear":15283.0,"score_lmetric":5990},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":113053,"ongoing_decode_tokens":0,"pending_prefill_tokens":92573,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":5990,"score_linear":113053.0,"score_lmetric":98563},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":5990,"score_linear":125063.0,"score_lmetric":246746},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":5990,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":22109,"ongoing_decode_tokens":0,"pending_prefill_tokens":605,"num_requests":1,"active_p_offloads":0,"cached_blocks":128,"cache_hit":0,"new_prefill":5990,"score_linear":22109.0,"score_lmetric":6595}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348399.221943056,"t_first_token_unix":1779779264.2358863,"t_done":348399.614592461,"t_done_unix":1779779264.6285355},{"request_id":"1262354:2:1263593:75","session_id":"1262354","input_length":22109,"t_proxy_recv":348398.713370952,"t_decision_unix":1779779263.727312,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":21504,"affinity_cache_ratio":0.9726355782712922,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":21504,"estimated_new_tokens":605,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":145,"cache_hit":0,"new_prefill":22109,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55989,"ongoing_decode_tokens":55989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":261,"cache_hit":0,"new_prefill":22109,"score_linear":55989.0,"score_lmetric":22109},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":68131,"ongoing_decode_tokens":68131,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":22109,"score_linear":68131.0,"score_lmetric":22109},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":22109,"score_linear":15283.0,"score_lmetric":22109},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":113053,"ongoing_decode_tokens":0,"pending_prefill_tokens":92573,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":22109,"score_linear":113053.0,"score_lmetric":114682},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":22109,"score_linear":125063.0,"score_lmetric":278984},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":22109,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":128,"cache_hit":21504,"new_prefill":605,"score_linear":-21504.0,"score_lmetric":0}],"chosen_score_linear":-21504.0,"chosen_score_lmetric":0,"t_first_token":348398.861721956,"t_first_token_unix":1779779263.8756657,"t_done":348400.097260203,"t_done_unix":1779779265.1112037},{"request_id":"1263971:1:1263971:80","session_id":"1263971","input_length":6336,"t_proxy_recv":348400.135451074,"t_decision_unix":1779779265.1493926,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":6336,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":6336,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55989,"ongoing_decode_tokens":55989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":261,"cache_hit":0,"new_prefill":6336,"score_linear":55989.0,"score_lmetric":6336},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":296,"cache_hit":0,"new_prefill":6336,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":71575,"ongoing_decode_tokens":15283,"pending_prefill_tokens":56292,"num_requests":2,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":6336,"score_linear":71575.0,"score_lmetric":125256},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":113053,"ongoing_decode_tokens":0,"pending_prefill_tokens":92573,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":6336,"score_linear":113053.0,"score_lmetric":98909},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":6336,"score_linear":125063.0,"score_lmetric":247438},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":50783,"ongoing_decode_tokens":0,"pending_prefill_tokens":50783,"num_requests":1,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":6336,"score_linear":50783.0,"score_lmetric":57119},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":129,"cache_hit":0,"new_prefill":6336,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348400.550270661,"t_first_token_unix":1779779265.5642142,"t_done":348400.916430044,"t_done_unix":1779779265.9303732},{"request_id":"1257323:1:1257323:48","session_id":"1257323","input_length":90488,"t_proxy_recv":348376.861010341,"t_decision_unix":1779779241.8749404,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":90488,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":104,"cache_hit":0,"new_prefill":90488,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":165,"cache_hit":0,"new_prefill":90488,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":162,"cache_hit":0,"new_prefill":90488,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":189,"cache_hit":0,"new_prefill":90488,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":90488,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":24,"cache_hit":0,"new_prefill":90488,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":169,"cache_hit":0,"new_prefill":90488,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":90488,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348404.49521728,"t_first_token_unix":1779779269.5091608,"t_done":348404.917595242,"t_done_unix":1779779269.931539},{"request_id":"1239034:3:1262346:68","session_id":"1239034","input_length":22565,"t_proxy_recv":348394.319259736,"t_decision_unix":1779779259.333201,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":20480,"affinity_cache_ratio":0.9076002658985154,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":20480,"estimated_new_tokens":2085,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":143,"cache_hit":0,"new_prefill":22565,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":29988,"ongoing_decode_tokens":0,"pending_prefill_tokens":2852,"num_requests":1,"active_p_offloads":0,"cached_blocks":256,"cache_hit":0,"new_prefill":22565,"score_linear":29988.0,"score_lmetric":25417},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":68131,"ongoing_decode_tokens":0,"pending_prefill_tokens":68131,"num_requests":1,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":22565,"score_linear":68131.0,"score_lmetric":90696},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":22565,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90488,"ongoing_decode_tokens":0,"pending_prefill_tokens":90488,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":20480,"new_prefill":2085,"score_linear":70008.0,"score_lmetric":92573},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":22565,"score_linear":125063.0,"score_lmetric":279896},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":197,"cache_hit":0,"new_prefill":22565,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":86,"cache_hit":0,"new_prefill":22565,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":70008.0,"chosen_score_lmetric":92573,"t_first_token":348404.496905557,"t_first_token_unix":1779779269.510849,"t_done":348405.044031493,"t_done_unix":1779779270.0579753},{"request_id":"1265000:1:1265000:83","session_id":"1265000","input_length":301,"t_proxy_recv":348403.760669888,"t_decision_unix":1779779268.7746112,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":301,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":110048,"ongoing_decode_tokens":0,"pending_prefill_tokens":110048,"num_requests":1,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":301,"score_linear":110048.0,"score_lmetric":110349},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55989,"ongoing_decode_tokens":55989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":261,"cache_hit":0,"new_prefill":301,"score_linear":55989.0,"score_lmetric":301},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":296,"cache_hit":0,"new_prefill":301,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":71575,"ongoing_decode_tokens":15283,"pending_prefill_tokens":56292,"num_requests":2,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":301,"score_linear":71575.0,"score_lmetric":113186},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":113053,"ongoing_decode_tokens":0,"pending_prefill_tokens":92573,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":301,"score_linear":113053.0,"score_lmetric":92874},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":301,"score_linear":125063.0,"score_lmetric":235368},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":50783,"ongoing_decode_tokens":0,"pending_prefill_tokens":50783,"num_requests":1,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":301,"score_linear":50783.0,"score_lmetric":51084},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":141,"cache_hit":0,"new_prefill":301,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348403.799853831,"t_first_token_unix":1779779268.8137968,"t_done":348405.316962386,"t_done_unix":1779779270.3309057},{"request_id":"1265128:1:1265128:85","session_id":"1265128","input_length":14028,"t_proxy_recv":348404.089734677,"t_decision_unix":1779779269.103676,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":14028,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":14028,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":110048,"ongoing_decode_tokens":0,"pending_prefill_tokens":110048,"num_requests":1,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":14028,"score_linear":110048.0,"score_lmetric":124076},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55989,"ongoing_decode_tokens":55989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":261,"cache_hit":0,"new_prefill":14028,"score_linear":55989.0,"score_lmetric":14028},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":301,"ongoing_decode_tokens":301,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":296,"cache_hit":0,"new_prefill":14028,"score_linear":301.0,"score_lmetric":14028},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":71575,"ongoing_decode_tokens":15283,"pending_prefill_tokens":56292,"num_requests":2,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":14028,"score_linear":71575.0,"score_lmetric":140640},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":113053,"ongoing_decode_tokens":0,"pending_prefill_tokens":92573,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":14028,"score_linear":113053.0,"score_lmetric":106601},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":14028,"score_linear":125063.0,"score_lmetric":262822},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":50783,"ongoing_decode_tokens":0,"pending_prefill_tokens":50783,"num_requests":1,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":14028,"score_linear":50783.0,"score_lmetric":64811},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":23540,"ongoing_decode_tokens":0,"pending_prefill_tokens":23540,"num_requests":1,"active_p_offloads":0,"cached_blocks":141,"cache_hit":0,"new_prefill":14028,"score_linear":23540.0,"score_lmetric":37568}],"chosen_score_linear":301.0,"chosen_score_lmetric":14028,"t_first_token":348405.298972089,"t_first_token_unix":1779779270.3129156,"t_done":348405.735317504,"t_done_unix":1779779270.7492604},{"request_id":"1266067:1:1266067:91","session_id":"1266067","input_length":579,"t_proxy_recv":348407.311692046,"t_decision_unix":1779779272.3256338,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.625,"fallback_score":579,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":579,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":579,"score_linear":135235.0,"score_lmetric":227596},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55989,"ongoing_decode_tokens":55989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":261,"cache_hit":0,"new_prefill":579,"score_linear":55989.0,"score_lmetric":579},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":13178,"ongoing_decode_tokens":0,"pending_prefill_tokens":13178,"num_requests":1,"active_p_offloads":0,"cached_blocks":323,"cache_hit":0,"new_prefill":579,"score_linear":13178.0,"score_lmetric":13757},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":92560,"ongoing_decode_tokens":15283,"pending_prefill_tokens":64989,"num_requests":3,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":579,"score_linear":92560.0,"score_lmetric":196704},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":31709,"ongoing_decode_tokens":31709,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":258,"cache_hit":0,"new_prefill":579,"score_linear":31709.0,"score_lmetric":579},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":579,"score_linear":125063.0,"score_lmetric":235924},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":50783,"ongoing_decode_tokens":0,"pending_prefill_tokens":50783,"num_requests":1,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":579,"score_linear":50783.0,"score_lmetric":51362},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":45884,"ongoing_decode_tokens":45884,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":141,"cache_hit":0,"new_prefill":579,"score_linear":45884.0,"score_lmetric":1158}],"chosen_score_linear":31709.0,"chosen_score_lmetric":579,"t_first_token":348407.488774484,"t_first_token_unix":1779779272.5027184,"t_done":348407.677870066,"t_done_unix":1779779272.6918135},{"request_id":"1263611:2:1265770:88","session_id":"1263611","input_length":13178,"t_proxy_recv":348406.409797691,"t_decision_unix":1779779271.423739,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":0,"chosen_idx":2,"affinity_cache_hit":5632,"affinity_cache_ratio":0.4273789649415693,"affinity_num_requests":2,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":13178,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":5632,"new_prefill":7546,"score_linear":129603.0,"score_lmetric":241530},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55989,"ongoing_decode_tokens":55989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":261,"cache_hit":0,"new_prefill":13178,"score_linear":55989.0,"score_lmetric":13178},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":323,"cache_hit":0,"new_prefill":13178,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":71575,"ongoing_decode_tokens":15283,"pending_prefill_tokens":56292,"num_requests":2,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":13178,"score_linear":71575.0,"score_lmetric":138940},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":258,"cache_hit":0,"new_prefill":13178,"score_linear":27639.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":13178,"score_linear":125063.0,"score_lmetric":261122},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":50783,"ongoing_decode_tokens":0,"pending_prefill_tokens":50783,"num_requests":1,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":13178,"score_linear":50783.0,"score_lmetric":63961},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":45884,"ongoing_decode_tokens":0,"pending_prefill_tokens":23868,"num_requests":2,"active_p_offloads":0,"cached_blocks":141,"cache_hit":0,"new_prefill":13178,"score_linear":45884.0,"score_lmetric":74092}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348407.514556712,"t_first_token_unix":1779779272.5285,"t_done":348407.765782487,"t_done_unix":1779779272.7797256},{"request_id":"1250772:3:1265019:84","session_id":"1250772","input_length":23540,"t_proxy_recv":348403.833222825,"t_decision_unix":1779779268.8471644,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":5,"chosen_idx":7,"affinity_cache_hit":10752,"affinity_cache_ratio":0.45675446049277824,"affinity_num_requests":2,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":23540,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":110048,"ongoing_decode_tokens":0,"pending_prefill_tokens":110048,"num_requests":1,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":23540,"score_linear":110048.0,"score_lmetric":133588},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55989,"ongoing_decode_tokens":55989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":261,"cache_hit":0,"new_prefill":23540,"score_linear":55989.0,"score_lmetric":23540},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":301,"ongoing_decode_tokens":301,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":296,"cache_hit":0,"new_prefill":23540,"score_linear":301.0,"score_lmetric":23540},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":71575,"ongoing_decode_tokens":15283,"pending_prefill_tokens":56292,"num_requests":2,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":23540,"score_linear":71575.0,"score_lmetric":159664},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":113053,"ongoing_decode_tokens":0,"pending_prefill_tokens":92573,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":23540,"score_linear":113053.0,"score_lmetric":116113},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":10752,"new_prefill":12788,"score_linear":114311.0,"score_lmetric":260342},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":50783,"ongoing_decode_tokens":0,"pending_prefill_tokens":50783,"num_requests":1,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":23540,"score_linear":50783.0,"score_lmetric":74323},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":141,"cache_hit":0,"new_prefill":23540,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348406.499187997,"t_first_token_unix":1779779271.5131314,"t_done":348408.128012922,"t_done_unix":1779779273.1419566},{"request_id":"1265942:1:1265942:89","session_id":"1265942","input_length":4070,"t_proxy_recv":348407.038910562,"t_decision_unix":1779779272.052852,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":4070,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":4070,"score_linear":135235.0,"score_lmetric":234578},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55989,"ongoing_decode_tokens":55989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":261,"cache_hit":0,"new_prefill":4070,"score_linear":55989.0,"score_lmetric":4070},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":13178,"ongoing_decode_tokens":0,"pending_prefill_tokens":13178,"num_requests":1,"active_p_offloads":0,"cached_blocks":323,"cache_hit":0,"new_prefill":4070,"score_linear":13178.0,"score_lmetric":17248},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":71575,"ongoing_decode_tokens":15283,"pending_prefill_tokens":56292,"num_requests":2,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":4070,"score_linear":71575.0,"score_lmetric":120724},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":258,"cache_hit":0,"new_prefill":4070,"score_linear":27639.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":4070,"score_linear":125063.0,"score_lmetric":242906},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":50783,"ongoing_decode_tokens":0,"pending_prefill_tokens":50783,"num_requests":1,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":4070,"score_linear":50783.0,"score_lmetric":54853},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":45884,"ongoing_decode_tokens":45884,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":141,"cache_hit":0,"new_prefill":4070,"score_linear":45884.0,"score_lmetric":8140}],"chosen_score_linear":27639.0,"chosen_score_lmetric":0,"t_first_token":348407.298713889,"t_first_token_unix":1779779272.3126574,"t_done":348408.158722441,"t_done_unix":1779779273.1726654},{"request_id":"1262354:3:1265178:86","session_id":"1262354","input_length":22344,"t_proxy_recv":348404.198994011,"t_decision_unix":1779779269.2129354,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":22016,"affinity_cache_ratio":0.9853204439670605,"affinity_num_requests":1,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":22016,"estimated_new_tokens":328,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":110048,"ongoing_decode_tokens":0,"pending_prefill_tokens":110048,"num_requests":1,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":22344,"score_linear":110048.0,"score_lmetric":132392},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55989,"ongoing_decode_tokens":55989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":261,"cache_hit":0,"new_prefill":22344,"score_linear":55989.0,"score_lmetric":22344},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":14329,"ongoing_decode_tokens":301,"pending_prefill_tokens":14028,"num_requests":2,"active_p_offloads":0,"cached_blocks":296,"cache_hit":0,"new_prefill":22344,"score_linear":14329.0,"score_lmetric":72744},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":71575,"ongoing_decode_tokens":15283,"pending_prefill_tokens":56292,"num_requests":2,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":22344,"score_linear":71575.0,"score_lmetric":157272},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":113053,"ongoing_decode_tokens":0,"pending_prefill_tokens":92573,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":22344,"score_linear":113053.0,"score_lmetric":114917},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":22344,"score_linear":125063.0,"score_lmetric":279454},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":50783,"ongoing_decode_tokens":0,"pending_prefill_tokens":50783,"num_requests":1,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":22344,"score_linear":50783.0,"score_lmetric":73127},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":23540,"ongoing_decode_tokens":0,"pending_prefill_tokens":23540,"num_requests":1,"active_p_offloads":0,"cached_blocks":141,"cache_hit":22016,"new_prefill":328,"score_linear":1524.0,"score_lmetric":23868}],"chosen_score_linear":1524.0,"chosen_score_lmetric":23868,"t_first_token":348406.5020628,"t_first_token_unix":1779779271.516006,"t_done":348408.926280955,"t_done_unix":1779779273.940224},{"request_id":"1266440:1:1266440:92","session_id":"1266440","input_length":5381,"t_proxy_recv":348408.387675151,"t_decision_unix":1779779273.4016166,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":5381,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":5381,"score_linear":135235.0,"score_lmetric":237200},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55989,"ongoing_decode_tokens":55989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":261,"cache_hit":0,"new_prefill":5381,"score_linear":55989.0,"score_lmetric":5381},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":348,"cache_hit":0,"new_prefill":5381,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":92560,"ongoing_decode_tokens":15283,"pending_prefill_tokens":64989,"num_requests":3,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":5381,"score_linear":92560.0,"score_lmetric":211110},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":267,"cache_hit":0,"new_prefill":5381,"score_linear":27639.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":5381,"score_linear":125063.0,"score_lmetric":245528},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":50783,"ongoing_decode_tokens":0,"pending_prefill_tokens":50783,"num_requests":1,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":5381,"score_linear":50783.0,"score_lmetric":56164},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":22344,"ongoing_decode_tokens":22344,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":187,"cache_hit":0,"new_prefill":5381,"score_linear":22344.0,"score_lmetric":5381}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348408.770509821,"t_first_token_unix":1779779273.784453,"t_done":348409.074261876,"t_done_unix":1779779274.088205},{"request_id":"1263727:1:1263727:77","session_id":"1263727","input_length":50783,"t_proxy_recv":348399.297473022,"t_decision_unix":1779779264.3114142,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":50783,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":5990,"ongoing_decode_tokens":5990,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":145,"cache_hit":0,"new_prefill":50783,"score_linear":5990.0,"score_lmetric":50783},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55989,"ongoing_decode_tokens":55989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":261,"cache_hit":0,"new_prefill":50783,"score_linear":55989.0,"score_lmetric":50783},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":68377,"ongoing_decode_tokens":68377,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":296,"cache_hit":0,"new_prefill":50783,"score_linear":68377.0,"score_lmetric":50783},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":50783,"score_linear":15283.0,"score_lmetric":50783},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":113053,"ongoing_decode_tokens":0,"pending_prefill_tokens":92573,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":50783,"score_linear":113053.0,"score_lmetric":143356},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":50783,"score_linear":125063.0,"score_lmetric":336332},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":50783,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":22109,"ongoing_decode_tokens":22109,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":128,"cache_hit":0,"new_prefill":50783,"score_linear":22109.0,"score_lmetric":50783}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348409.010245746,"t_first_token_unix":1779779274.024189,"t_done":348409.197652895,"t_done_unix":1779779274.2115958},{"request_id":"1253804:4:1262690:70","session_id":"1253804","input_length":55989,"t_proxy_recv":348395.449718231,"t_decision_unix":1779779260.4636593,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":54784,"affinity_cache_ratio":0.978477915304792,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":54784,"estimated_new_tokens":1205,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":143,"cache_hit":0,"new_prefill":55989,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":261,"cache_hit":54784,"new_prefill":1205,"score_linear":-54784.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":68131,"ongoing_decode_tokens":0,"pending_prefill_tokens":68131,"num_requests":1,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":55989,"score_linear":68131.0,"score_lmetric":124120},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":231,"cache_hit":3584,"new_prefill":52405,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":113053,"ongoing_decode_tokens":0,"pending_prefill_tokens":92573,"num_requests":2,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":55989,"score_linear":113053.0,"score_lmetric":297124},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":55989,"score_linear":125063.0,"score_lmetric":346744},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":197,"cache_hit":0,"new_prefill":55989,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":21510,"ongoing_decode_tokens":0,"pending_prefill_tokens":21510,"num_requests":1,"active_p_offloads":0,"cached_blocks":86,"cache_hit":0,"new_prefill":55989,"score_linear":21510.0,"score_lmetric":77499}],"chosen_score_linear":-54784.0,"chosen_score_lmetric":0,"t_first_token":348395.868227965,"t_first_token_unix":1779779260.8821714,"t_done":348409.32006035,"t_done_unix":1779779274.334004},{"request_id":"1263971:2:1266797:94","session_id":"1263971","input_length":9504,"t_proxy_recv":348409.733929647,"t_decision_unix":1779779274.747871,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":6144,"affinity_cache_ratio":0.6464646464646465,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":6144,"estimated_new_tokens":3360,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":9504,"score_linear":135235.0,"score_lmetric":245446},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":9504,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":9504,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":92560,"ongoing_decode_tokens":15283,"pending_prefill_tokens":64989,"num_requests":3,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":9504,"score_linear":92560.0,"score_lmetric":223479},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":267,"cache_hit":0,"new_prefill":9504,"score_linear":27639.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":9504,"score_linear":125063.0,"score_lmetric":253774},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":306,"cache_hit":0,"new_prefill":9504,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":17466,"ongoing_decode_tokens":0,"pending_prefill_tokens":17466,"num_requests":1,"active_p_offloads":0,"cached_blocks":188,"cache_hit":6144,"new_prefill":3360,"score_linear":11322.0,"score_lmetric":20826}],"chosen_score_linear":11322.0,"chosen_score_lmetric":20826,"t_first_token":348411.236093714,"t_first_token_unix":1779779276.2500367,"t_done":348411.428792056,"t_done_unix":1779779276.4427354},{"request_id":"1260792:2:1265951:90","session_id":"1260792","input_length":20985,"t_proxy_recv":348407.06485419,"t_decision_unix":1779779272.0787957,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":12288,"affinity_cache_ratio":0.5855611150822015,"affinity_num_requests":2,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":12288,"estimated_new_tokens":8697,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":20985,"score_linear":135235.0,"score_lmetric":268408},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55989,"ongoing_decode_tokens":55989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":261,"cache_hit":0,"new_prefill":20985,"score_linear":55989.0,"score_lmetric":20985},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":13178,"ongoing_decode_tokens":0,"pending_prefill_tokens":13178,"num_requests":1,"active_p_offloads":0,"cached_blocks":323,"cache_hit":0,"new_prefill":20985,"score_linear":13178.0,"score_lmetric":34163},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":71575,"ongoing_decode_tokens":15283,"pending_prefill_tokens":56292,"num_requests":2,"active_p_offloads":0,"cached_blocks":231,"cache_hit":12288,"new_prefill":8697,"score_linear":59287.0,"score_lmetric":129978},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":31709,"ongoing_decode_tokens":27639,"pending_prefill_tokens":4070,"num_requests":1,"active_p_offloads":0,"cached_blocks":258,"cache_hit":0,"new_prefill":20985,"score_linear":31709.0,"score_lmetric":25055},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":20985,"score_linear":125063.0,"score_lmetric":276736},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":50783,"ongoing_decode_tokens":0,"pending_prefill_tokens":50783,"num_requests":1,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":20985,"score_linear":50783.0,"score_lmetric":71768},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":45884,"ongoing_decode_tokens":45884,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":141,"cache_hit":0,"new_prefill":20985,"score_linear":45884.0,"score_lmetric":41970}],"chosen_score_linear":59287.0,"chosen_score_lmetric":129978,"t_first_token":348411.984647073,"t_first_token_unix":1779779276.9985907,"t_done":348412.50253305,"t_done_unix":1779779277.5164769},{"request_id":"1263749:1:1263749:79","session_id":"1263749","input_length":56292,"t_proxy_recv":348399.370238795,"t_decision_unix":1779779264.3841803,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":56292,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":56292,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":5990,"ongoing_decode_tokens":5990,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":145,"cache_hit":0,"new_prefill":56292,"score_linear":5990.0,"score_lmetric":56292},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55989,"ongoing_decode_tokens":55989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":261,"cache_hit":0,"new_prefill":56292,"score_linear":55989.0,"score_lmetric":56292},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":68377,"ongoing_decode_tokens":68377,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":296,"cache_hit":0,"new_prefill":56292,"score_linear":68377.0,"score_lmetric":56292},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":56292,"score_linear":15283.0,"score_lmetric":56292},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":113053,"ongoing_decode_tokens":0,"pending_prefill_tokens":92573,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":56292,"score_linear":113053.0,"score_lmetric":148865},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":56292,"score_linear":125063.0,"score_lmetric":347350},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":50783,"ongoing_decode_tokens":0,"pending_prefill_tokens":50783,"num_requests":1,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":56292,"score_linear":50783.0,"score_lmetric":107075},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":22109,"ongoing_decode_tokens":22109,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":128,"cache_hit":0,"new_prefill":56292,"score_linear":22109.0,"score_lmetric":56292}],"chosen_score_linear":15283.0,"chosen_score_lmetric":56292,"t_first_token":348410.814692606,"t_first_token_unix":1779779275.8286364,"t_done":348413.445680974,"t_done_unix":1779779278.4596243},{"request_id":"1265942:2:1268069:97","session_id":"1265942","input_length":10457,"t_proxy_recv":348414.04333992,"t_decision_unix":1779779279.0572815,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":4096,"affinity_cache_ratio":0.39169934015492014,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":4096,"estimated_new_tokens":6361,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":10457,"score_linear":135235.0,"score_lmetric":247352},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":10457,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":10457,"score_linear":126924.0,"score_lmetric":118437},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":10457,"score_linear":15283.0,"score_lmetric":10457},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":267,"cache_hit":4096,"new_prefill":6361,"score_linear":23543.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":10457,"score_linear":125063.0,"score_lmetric":255680},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":306,"cache_hit":0,"new_prefill":10457,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":46137,"ongoing_decode_tokens":17466,"pending_prefill_tokens":6143,"num_requests":2,"active_p_offloads":0,"cached_blocks":194,"cache_hit":0,"new_prefill":10457,"score_linear":46137.0,"score_lmetric":33200}],"chosen_score_linear":23543.0,"chosen_score_lmetric":0,"t_first_token":348414.62023027,"t_first_token_unix":1779779279.6341734,"t_done":348415.15234966,"t_done_unix":1779779280.166293},{"request_id":"1262354:4:1267821:95","session_id":"1262354","input_length":28671,"t_proxy_recv":348413.3441109,"t_decision_unix":1779779278.3580518,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":22528,"affinity_cache_ratio":0.7857416902096195,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":22528,"estimated_new_tokens":6143,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":28671,"score_linear":135235.0,"score_lmetric":283780},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":28671,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":28671,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":71575,"ongoing_decode_tokens":71575,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":248,"cache_hit":0,"new_prefill":28671,"score_linear":71575.0,"score_lmetric":57342},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":267,"cache_hit":0,"new_prefill":28671,"score_linear":27639.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":28671,"score_linear":125063.0,"score_lmetric":292108},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":306,"cache_hit":0,"new_prefill":28671,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":17466,"ongoing_decode_tokens":17466,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":194,"cache_hit":22528,"new_prefill":6143,"score_linear":-5062.0,"score_lmetric":6143}],"chosen_score_linear":-5062.0,"chosen_score_lmetric":6143,"t_first_token":348414.528913826,"t_first_token_unix":1779779279.542857,"t_done":348416.131836513,"t_done_unix":1779779281.1457803},{"request_id":"1268634:1:1268634:99","session_id":"1268634","input_length":2298,"t_proxy_recv":348416.297786649,"t_decision_unix":1779779281.3117282,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":2298,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":2298,"score_linear":135235.0,"score_lmetric":231034},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":2298,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":2298,"score_linear":126924.0,"score_lmetric":110278},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":2298,"score_linear":15283.0,"score_lmetric":2298},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":279,"cache_hit":0,"new_prefill":2298,"score_linear":27639.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":2298,"score_linear":125063.0,"score_lmetric":239362},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":306,"cache_hit":0,"new_prefill":2298,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":17466,"ongoing_decode_tokens":17466,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":206,"cache_hit":0,"new_prefill":2298,"score_linear":17466.0,"score_lmetric":2298}],"chosen_score_linear":27639.0,"chosen_score_lmetric":0,"t_first_token":348416.513842204,"t_first_token_unix":1779779281.527786,"t_done":348416.688510492,"t_done_unix":1779779281.7024543},{"request_id":"1266668:1:1266668:93","session_id":"1266668","input_length":17466,"t_proxy_recv":348409.285260277,"t_decision_unix":1779779274.299202,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":17466,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":4096,"new_prefill":13370,"score_linear":131139.0,"score_lmetric":253178},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55989,"ongoing_decode_tokens":55989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":261,"cache_hit":0,"new_prefill":17466,"score_linear":55989.0,"score_lmetric":17466},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":17466,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":92560,"ongoing_decode_tokens":15283,"pending_prefill_tokens":64989,"num_requests":3,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":17466,"score_linear":92560.0,"score_lmetric":247365},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":267,"cache_hit":0,"new_prefill":17466,"score_linear":27639.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":17466,"score_linear":125063.0,"score_lmetric":269698},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":306,"cache_hit":0,"new_prefill":17466,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":188,"cache_hit":0,"new_prefill":17466,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348411.234026369,"t_first_token_unix":1779779276.2479699,"t_done":348416.765698971,"t_done_unix":1779779281.7796423},{"request_id":"1268649:1:1268649:100","session_id":"1268649","input_length":18638,"t_proxy_recv":348416.301317612,"t_decision_unix":1779779281.3152592,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":18638,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":18638,"score_linear":135235.0,"score_lmetric":263714},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":18638,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":18638,"score_linear":126924.0,"score_lmetric":126618},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":18638,"score_linear":15283.0,"score_lmetric":18638},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":29937,"ongoing_decode_tokens":27639,"pending_prefill_tokens":2298,"num_requests":1,"active_p_offloads":0,"cached_blocks":279,"cache_hit":0,"new_prefill":18638,"score_linear":29937.0,"score_lmetric":20936},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":18638,"score_linear":125063.0,"score_lmetric":272042},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":306,"cache_hit":0,"new_prefill":18638,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":17466,"ongoing_decode_tokens":17466,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":206,"cache_hit":0,"new_prefill":18638,"score_linear":17466.0,"score_lmetric":18638}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348418.254032915,"t_first_token_unix":1779779283.267977,"t_done":348418.398125323,"t_done_unix":1779779283.4120688},{"request_id":"1263971:3:1269088:104","session_id":"1263971","input_length":13955,"t_proxy_recv":348417.675284199,"t_decision_unix":1779779282.6892254,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":9216,"affinity_cache_ratio":0.660408455750627,"affinity_num_requests":2,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":9216,"estimated_new_tokens":4739,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":13955,"score_linear":135235.0,"score_lmetric":254348},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":13955,"score_linear":102074.0,"score_lmetric":112445},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":13955,"score_linear":126924.0,"score_lmetric":121935},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":13955,"score_linear":15283.0,"score_lmetric":13955},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":39834,"ongoing_decode_tokens":27639,"pending_prefill_tokens":12195,"num_requests":1,"active_p_offloads":0,"cached_blocks":283,"cache_hit":0,"new_prefill":13955,"score_linear":39834.0,"score_lmetric":26150},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":13955,"score_linear":125063.0,"score_lmetric":262676},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":18638,"ongoing_decode_tokens":0,"pending_prefill_tokens":18638,"num_requests":1,"active_p_offloads":0,"cached_blocks":306,"cache_hit":0,"new_prefill":13955,"score_linear":18638.0,"score_lmetric":32593},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":36349,"ongoing_decode_tokens":0,"pending_prefill_tokens":12797,"num_requests":2,"active_p_offloads":0,"cached_blocks":241,"cache_hit":9216,"new_prefill":4739,"score_linear":27133.0,"score_lmetric":35072}],"chosen_score_linear":27133.0,"chosen_score_lmetric":35072,"t_first_token":348418.480528703,"t_first_token_unix":1779779283.4944723,"t_done":348418.737993654,"t_done_unix":1779779283.7519374},{"request_id":"1268831:1:1268831:101","session_id":"1268831","input_length":12195,"t_proxy_recv":348416.778729923,"t_decision_unix":1779779281.7926712,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":12195,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":12195,"score_linear":135235.0,"score_lmetric":250828},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":12195,"score_linear":102074.0,"score_lmetric":110685},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":12195,"score_linear":126924.0,"score_lmetric":120175},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":12195,"score_linear":15283.0,"score_lmetric":12195},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":283,"cache_hit":0,"new_prefill":12195,"score_linear":27639.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":12195,"score_linear":125063.0,"score_lmetric":259156},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":18638,"ongoing_decode_tokens":0,"pending_prefill_tokens":18638,"num_requests":1,"active_p_offloads":0,"cached_blocks":306,"cache_hit":0,"new_prefill":12195,"score_linear":18638.0,"score_lmetric":30833},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":241,"cache_hit":0,"new_prefill":12195,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":27639.0,"chosen_score_lmetric":0,"t_first_token":348417.790813781,"t_first_token_unix":1779779282.8047574,"t_done":348419.744679021,"t_done_unix":1779779284.7586224},{"request_id":"1250772:4:1268869:103","session_id":"1250772","input_length":24029,"t_proxy_recv":348416.936253429,"t_decision_unix":1779779281.9501948,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":23552,"affinity_cache_ratio":0.9801489866411419,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":23552,"estimated_new_tokens":477,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":24029,"score_linear":135235.0,"score_lmetric":274496},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":24029,"score_linear":102074.0,"score_lmetric":122519},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":24029,"score_linear":126924.0,"score_lmetric":132009},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":24029,"score_linear":15283.0,"score_lmetric":24029},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":39834,"ongoing_decode_tokens":27639,"pending_prefill_tokens":12195,"num_requests":1,"active_p_offloads":0,"cached_blocks":283,"cache_hit":0,"new_prefill":24029,"score_linear":39834.0,"score_lmetric":36224},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":10752,"new_prefill":13277,"score_linear":114311.0,"score_lmetric":261320},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":18638,"ongoing_decode_tokens":0,"pending_prefill_tokens":18638,"num_requests":1,"active_p_offloads":0,"cached_blocks":306,"cache_hit":0,"new_prefill":24029,"score_linear":18638.0,"score_lmetric":42667},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":12320,"ongoing_decode_tokens":0,"pending_prefill_tokens":12320,"num_requests":1,"active_p_offloads":0,"cached_blocks":241,"cache_hit":23552,"new_prefill":477,"score_linear":-11232.0,"score_lmetric":12797}],"chosen_score_linear":-11232.0,"chosen_score_lmetric":12797,"t_first_token":348417.97932677,"t_first_token_unix":1779779282.99327,"t_done":348420.293686889,"t_done_unix":1779779285.3076305},{"request_id":"1269901:1:1269901:107","session_id":"1269901","input_length":1323,"t_proxy_recv":348420.312906358,"t_decision_unix":1779779285.326848,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":1323,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":1323,"score_linear":135235.0,"score_lmetric":229084},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":1323,"score_linear":102074.0,"score_lmetric":99813},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":1323,"score_linear":126924.0,"score_lmetric":109303},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":1323,"score_linear":15283.0,"score_lmetric":1323},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":307,"cache_hit":0,"new_prefill":1323,"score_linear":27639.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":1323,"score_linear":125063.0,"score_lmetric":237412},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":28906,"ongoing_decode_tokens":0,"pending_prefill_tokens":28906,"num_requests":1,"active_p_offloads":0,"cached_blocks":342,"cache_hit":0,"new_prefill":1323,"score_linear":28906.0,"score_lmetric":30229},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":26424,"ongoing_decode_tokens":26424,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":251,"cache_hit":0,"new_prefill":1323,"score_linear":26424.0,"score_lmetric":2646}],"chosen_score_linear":27639.0,"chosen_score_lmetric":0,"t_first_token":348420.505346899,"t_first_token_unix":1779779285.5192904,"t_done":348421.239190912,"t_done_unix":1779779286.2531343},{"request_id":"1260327:2:1270005:108","session_id":"1260327","input_length":30828,"t_proxy_recv":348420.739657997,"t_decision_unix":1779779285.7535994,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":30720,"affinity_cache_ratio":0.9964966913195796,"affinity_num_requests":2,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":30720,"estimated_new_tokens":108,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":30828,"score_linear":135235.0,"score_lmetric":288094},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":30828,"score_linear":102074.0,"score_lmetric":129318},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":30828,"score_linear":126924.0,"score_lmetric":138808},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":30828,"score_linear":15283.0,"score_lmetric":30828},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":28962,"ongoing_decode_tokens":28962,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":307,"cache_hit":0,"new_prefill":30828,"score_linear":28962.0,"score_lmetric":30828},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":30828,"score_linear":125063.0,"score_lmetric":296422},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":28906,"ongoing_decode_tokens":0,"pending_prefill_tokens":28906,"num_requests":1,"active_p_offloads":0,"cached_blocks":342,"cache_hit":0,"new_prefill":30828,"score_linear":28906.0,"score_lmetric":59734},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":26424,"ongoing_decode_tokens":26424,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":251,"cache_hit":30720,"new_prefill":108,"score_linear":-4296.0,"score_lmetric":216}],"chosen_score_linear":-4296.0,"chosen_score_lmetric":216,"t_first_token":348420.823940122,"t_first_token_unix":1779779285.8378835,"t_done":348421.354162224,"t_done_unix":1779779286.3681056},{"request_id":"1269373:1:1269373:106","session_id":"1269373","input_length":14104,"t_proxy_recv":348418.756619746,"t_decision_unix":1779779283.7705607,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.375,"fallback_score":12848,"tie_break_used":false,"cache_hit":7680,"estimated_new_tokens":6424,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":4096,"new_prefill":10008,"score_linear":131139.0,"score_lmetric":246454},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":14104,"score_linear":102074.0,"score_lmetric":112594},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":14104,"score_linear":126924.0,"score_lmetric":122084},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":14104,"score_linear":15283.0,"score_lmetric":14104},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":39834,"ongoing_decode_tokens":39834,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":283,"cache_hit":0,"new_prefill":14104,"score_linear":39834.0,"score_lmetric":14104},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":14104,"score_linear":125063.0,"score_lmetric":262974},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":28906,"ongoing_decode_tokens":0,"pending_prefill_tokens":28906,"num_requests":1,"active_p_offloads":0,"cached_blocks":342,"cache_hit":0,"new_prefill":14104,"score_linear":28906.0,"score_lmetric":43010},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":36349,"ongoing_decode_tokens":36349,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":250,"cache_hit":7680,"new_prefill":6424,"score_linear":28669.0,"score_lmetric":12848}],"chosen_score_linear":28669.0,"chosen_score_lmetric":12848,"t_first_token":348419.475051664,"t_first_token_unix":1779779284.4889956,"t_done":348421.50811667,"t_done_unix":1779779286.5220604},{"request_id":"1268861:1:1268861:102","session_id":"1268861","input_length":12320,"t_proxy_recv":348416.897206706,"t_decision_unix":1779779281.9111478,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":12320,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":12320,"score_linear":135235.0,"score_lmetric":251078},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":12320,"score_linear":102074.0,"score_lmetric":110810},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":12320,"score_linear":126924.0,"score_lmetric":120300},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":12320,"score_linear":15283.0,"score_lmetric":12320},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":39834,"ongoing_decode_tokens":27639,"pending_prefill_tokens":12195,"num_requests":1,"active_p_offloads":0,"cached_blocks":283,"cache_hit":0,"new_prefill":12320,"score_linear":39834.0,"score_lmetric":24515},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":12320,"score_linear":125063.0,"score_lmetric":259406},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":18638,"ongoing_decode_tokens":0,"pending_prefill_tokens":18638,"num_requests":1,"active_p_offloads":0,"cached_blocks":306,"cache_hit":0,"new_prefill":12320,"score_linear":18638.0,"score_lmetric":30958},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":241,"cache_hit":0,"new_prefill":12320,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348417.978038461,"t_first_token_unix":1779779282.9919827,"t_done":348422.140204613,"t_done_unix":1779779287.1541479},{"request_id":"1256273:4:1259632:60","session_id":"1256273","input_length":12270,"t_proxy_recv":348384.781768957,"t_decision_unix":1779779249.7957103,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":7680,"affinity_cache_ratio":0.6259168704156479,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":7680,"estimated_new_tokens":4590,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":114,"cache_hit":0,"new_prefill":12270,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":80365,"ongoing_decode_tokens":27387,"pending_prefill_tokens":17650,"num_requests":2,"active_p_offloads":0,"cached_blocks":165,"cache_hit":0,"new_prefill":12270,"score_linear":80365.0,"score_lmetric":59840},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":68131,"ongoing_decode_tokens":0,"pending_prefill_tokens":68131,"num_requests":1,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":12270,"score_linear":68131.0,"score_lmetric":80401},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":9250,"ongoing_decode_tokens":9250,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":189,"cache_hit":0,"new_prefill":12270,"score_linear":9250.0,"score_lmetric":12270},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90488,"ongoing_decode_tokens":0,"pending_prefill_tokens":90488,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":12270,"score_linear":90488.0,"score_lmetric":102758},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":112793,"ongoing_decode_tokens":0,"pending_prefill_tokens":112793,"num_requests":1,"active_p_offloads":0,"cached_blocks":36,"cache_hit":7680,"new_prefill":4590,"score_linear":105113.0,"score_lmetric":117383},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":14291,"ongoing_decode_tokens":14291,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":169,"cache_hit":0,"new_prefill":12270,"score_linear":14291.0,"score_lmetric":12270},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":12270,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":105113.0,"chosen_score_lmetric":117383,"t_first_token":348423.754777874,"t_first_token_unix":1779779288.768721,"t_done":348424.812224333,"t_done_unix":1779779289.8261673},{"request_id":"1258859:1:1258859:57","session_id":"1258859","input_length":112793,"t_proxy_recv":348382.193540376,"t_decision_unix":1779779247.2074816,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":112793,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":114,"cache_hit":0,"new_prefill":112793,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":80365,"ongoing_decode_tokens":27387,"pending_prefill_tokens":17650,"num_requests":2,"active_p_offloads":0,"cached_blocks":165,"cache_hit":0,"new_prefill":112793,"score_linear":80365.0,"score_lmetric":260886},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":68131,"ongoing_decode_tokens":0,"pending_prefill_tokens":68131,"num_requests":1,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":112793,"score_linear":68131.0,"score_lmetric":180924},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":189,"cache_hit":0,"new_prefill":112793,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90488,"ongoing_decode_tokens":0,"pending_prefill_tokens":90488,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":112793,"score_linear":90488.0,"score_lmetric":203281},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":112793,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":169,"cache_hit":0,"new_prefill":112793,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":26,"cache_hit":0,"new_prefill":112793,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348423.523009182,"t_first_token_unix":1779779288.5369525,"t_done":348424.927620878,"t_done_unix":1779779289.941564},{"request_id":"1263727:2:1270052:109","session_id":"1263727","input_length":57803,"t_proxy_recv":348420.933825757,"t_decision_unix":1779779285.947767,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":50688,"affinity_cache_ratio":0.876909502966974,"affinity_num_requests":1,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":50688,"estimated_new_tokens":7115,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":57803,"score_linear":135235.0,"score_lmetric":342044},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":57803,"score_linear":102074.0,"score_lmetric":156293},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":57803,"score_linear":126924.0,"score_lmetric":165783},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":57803,"score_linear":15283.0,"score_lmetric":57803},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":28962,"ongoing_decode_tokens":28962,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":307,"cache_hit":0,"new_prefill":57803,"score_linear":28962.0,"score_lmetric":57803},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":57803,"score_linear":125063.0,"score_lmetric":350372},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":28906,"ongoing_decode_tokens":0,"pending_prefill_tokens":28906,"num_requests":1,"active_p_offloads":0,"cached_blocks":342,"cache_hit":50688,"new_prefill":7115,"score_linear":-21782.0,"score_lmetric":36021},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":57252,"ongoing_decode_tokens":57252,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":251,"cache_hit":0,"new_prefill":57803,"score_linear":57252.0,"score_lmetric":173409}],"chosen_score_linear":-21782.0,"chosen_score_lmetric":36021,"t_first_token":348424.831258554,"t_first_token_unix":1779779289.8452017,"t_done":348425.07983945,"t_done_unix":1779779290.0937831},{"request_id":"1266440:2:1270501:112","session_id":"1266440","input_length":11128,"t_proxy_recv":348422.508096399,"t_decision_unix":1779779287.5220377,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":2,"chosen_idx":7,"affinity_cache_hit":5120,"affinity_cache_ratio":0.46010064701653486,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":11128,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":11128,"score_linear":135235.0,"score_lmetric":248694},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":11128,"score_linear":102074.0,"score_lmetric":109618},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":5120,"new_prefill":6008,"score_linear":121804.0,"score_lmetric":113988},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":11128,"score_linear":15283.0,"score_lmetric":11128},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":55566,"ongoing_decode_tokens":27639,"pending_prefill_tokens":27927,"num_requests":1,"active_p_offloads":0,"cached_blocks":309,"cache_hit":0,"new_prefill":11128,"score_linear":55566.0,"score_lmetric":39055},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":11128,"score_linear":125063.0,"score_lmetric":257022},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":86709,"ongoing_decode_tokens":28906,"pending_prefill_tokens":7115,"num_requests":2,"active_p_offloads":0,"cached_blocks":342,"cache_hit":0,"new_prefill":11128,"score_linear":86709.0,"score_lmetric":36486},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":288,"cache_hit":0,"new_prefill":11128,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348423.431328978,"t_first_token_unix":1779779288.4452722,"t_done":348425.502198438,"t_done_unix":1779779290.5161438},{"request_id":"1270340:1:1270340:111","session_id":"1270340","input_length":27927,"t_proxy_recv":348421.941466486,"t_decision_unix":1779779286.9554076,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":27927,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":27927,"score_linear":135235.0,"score_lmetric":282292},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":27927,"score_linear":102074.0,"score_lmetric":126417},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":27927,"score_linear":126924.0,"score_lmetric":135907},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":27927,"score_linear":15283.0,"score_lmetric":27927},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":309,"cache_hit":0,"new_prefill":27927,"score_linear":27639.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":27927,"score_linear":125063.0,"score_lmetric":290620},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":86709,"ongoing_decode_tokens":0,"pending_prefill_tokens":36021,"num_requests":2,"active_p_offloads":0,"cached_blocks":342,"cache_hit":0,"new_prefill":27927,"score_linear":86709.0,"score_lmetric":127896},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":12320,"ongoing_decode_tokens":12320,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":264,"cache_hit":0,"new_prefill":27927,"score_linear":12320.0,"score_lmetric":27927}],"chosen_score_linear":27639.0,"chosen_score_lmetric":0,"t_first_token":348425.527156215,"t_first_token_unix":1779779290.5410995,"t_done":348426.927859403,"t_done_unix":1779779291.9418025},{"request_id":"1262354:5:1269336:105","session_id":"1262354","input_length":28906,"t_proxy_recv":348418.63126521,"t_decision_unix":1779779283.6452062,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":7,"chosen_idx":6,"affinity_cache_hit":28672,"affinity_cache_ratio":0.9919047948522798,"affinity_num_requests":3,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":28906,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":28906,"score_linear":135235.0,"score_lmetric":284250},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":28906,"score_linear":102074.0,"score_lmetric":127396},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":28906,"score_linear":126924.0,"score_lmetric":136886},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":28906,"score_linear":15283.0,"score_lmetric":28906},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":39834,"ongoing_decode_tokens":39834,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":283,"cache_hit":0,"new_prefill":28906,"score_linear":39834.0,"score_lmetric":28906},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":28906,"score_linear":125063.0,"score_lmetric":292578},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":342,"cache_hit":0,"new_prefill":28906,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":50304,"ongoing_decode_tokens":50304,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":241,"cache_hit":28672,"new_prefill":234,"score_linear":21632.0,"score_lmetric":702}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348422.418310192,"t_first_token_unix":1779779287.4322536,"t_done":348427.260396599,"t_done_unix":1779779292.2743397},{"request_id":"1271847:1:1271847:123","session_id":"1271847","input_length":2524,"t_proxy_recv":348427.196359899,"t_decision_unix":1779779292.2103016,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.0,"fallback_score":5048,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":2524,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":2524,"score_linear":135235.0,"score_lmetric":231486},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":2524,"score_linear":102074.0,"score_lmetric":101014},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":2524,"score_linear":150574.0,"score_lmetric":240660},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":108713,"ongoing_decode_tokens":15283,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":2524,"score_linear":108713.0,"score_lmetric":191908},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":75410,"ongoing_decode_tokens":75410,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":363,"cache_hit":0,"new_prefill":2524,"score_linear":75410.0,"score_lmetric":5048},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":113923,"ongoing_decode_tokens":113923,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":2524,"score_linear":113923.0,"score_lmetric":5048},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":46612,"ongoing_decode_tokens":28906,"pending_prefill_tokens":17706,"num_requests":2,"active_p_offloads":0,"cached_blocks":355,"cache_hit":0,"new_prefill":2524,"score_linear":46612.0,"score_lmetric":40460},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":64508,"ongoing_decode_tokens":28385,"pending_prefill_tokens":21787,"num_requests":3,"active_p_offloads":0,"cached_blocks":309,"cache_hit":0,"new_prefill":2524,"score_linear":64508.0,"score_lmetric":72933}],"chosen_score_linear":113923.0,"chosen_score_lmetric":5048,"t_first_token":348427.539194817,"t_first_token_unix":1779779292.553138,"t_done":348427.883993162,"t_done_unix":1779779292.8979366},{"request_id":"1271991:1:1271991:124","session_id":"1271991","input_length":2851,"t_proxy_recv":348427.726713828,"t_decision_unix":1779779292.7406547,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.125,"fallback_score":5702,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":2851,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":2851,"score_linear":135235.0,"score_lmetric":232140},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":2851,"score_linear":102074.0,"score_lmetric":101341},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":2851,"score_linear":150574.0,"score_lmetric":241314},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":108713,"ongoing_decode_tokens":15283,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":2851,"score_linear":108713.0,"score_lmetric":192562},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":75410,"ongoing_decode_tokens":75410,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":363,"cache_hit":0,"new_prefill":2851,"score_linear":75410.0,"score_lmetric":5702},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":116447,"ongoing_decode_tokens":116447,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":2851,"score_linear":116447.0,"score_lmetric":8553},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":50480,"ongoing_decode_tokens":17706,"pending_prefill_tokens":4102,"num_requests":2,"active_p_offloads":0,"cached_blocks":411,"cache_hit":0,"new_prefill":2851,"score_linear":50480.0,"score_lmetric":13906},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":64508,"ongoing_decode_tokens":28385,"pending_prefill_tokens":21787,"num_requests":3,"active_p_offloads":0,"cached_blocks":309,"cache_hit":0,"new_prefill":2851,"score_linear":64508.0,"score_lmetric":73914}],"chosen_score_linear":75410.0,"chosen_score_lmetric":5702,"t_first_token":348427.93844152,"t_first_token_unix":1779779292.9523847,"t_done":348428.300962088,"t_done_unix":1779779293.3149054},{"request_id":"1271459:1:1271459:118","session_id":"1271459","input_length":751,"t_proxy_recv":348425.899989509,"t_decision_unix":1779779290.913931,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.5,"fallback_score":751,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":751,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":751,"score_linear":135235.0,"score_lmetric":227940},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":751,"score_linear":102074.0,"score_lmetric":99241},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":751,"score_linear":126924.0,"score_lmetric":108731},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":108713,"ongoing_decode_tokens":15283,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":751,"score_linear":108713.0,"score_lmetric":188362},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":75014,"ongoing_decode_tokens":55566,"pending_prefill_tokens":7160,"num_requests":2,"active_p_offloads":0,"cached_blocks":309,"cache_hit":0,"new_prefill":751,"score_linear":75014.0,"score_lmetric":15822},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":113172,"ongoing_decode_tokens":113172,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":751,"score_linear":113172.0,"score_lmetric":751},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":46612,"ongoing_decode_tokens":28906,"pending_prefill_tokens":17706,"num_requests":2,"active_p_offloads":0,"cached_blocks":355,"cache_hit":0,"new_prefill":751,"score_linear":46612.0,"score_lmetric":36914},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":28385,"ongoing_decode_tokens":28385,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":309,"cache_hit":0,"new_prefill":751,"score_linear":28385.0,"score_lmetric":751}],"chosen_score_linear":113172.0,"chosen_score_lmetric":751,"t_first_token":348426.173825764,"t_first_token_unix":1779779291.187769,"t_done":348428.640247652,"t_done_unix":1779779293.6541905},{"request_id":"1271328:1:1271328:117","session_id":"1271328","input_length":17706,"t_proxy_recv":348425.483321943,"t_decision_unix":1779779290.4972634,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.5,"fallback_score":17706,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":17706,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":17706,"score_linear":135235.0,"score_lmetric":261850},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":17706,"score_linear":102074.0,"score_lmetric":116196},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":17706,"score_linear":126924.0,"score_lmetric":125686},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":108713,"ongoing_decode_tokens":15283,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":17706,"score_linear":108713.0,"score_lmetric":222272},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":75014,"ongoing_decode_tokens":27639,"pending_prefill_tokens":35087,"num_requests":2,"active_p_offloads":0,"cached_blocks":309,"cache_hit":0,"new_prefill":17706,"score_linear":75014.0,"score_lmetric":105586},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":113172,"ongoing_decode_tokens":113172,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":17706,"score_linear":113172.0,"score_lmetric":17706},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":28906,"ongoing_decode_tokens":28906,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":355,"cache_hit":0,"new_prefill":17706,"score_linear":28906.0,"score_lmetric":17706},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":39513,"ongoing_decode_tokens":39513,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":288,"cache_hit":0,"new_prefill":17706,"score_linear":39513.0,"score_lmetric":35412}],"chosen_score_linear":28906.0,"chosen_score_lmetric":17706,"t_first_token":348427.236819114,"t_first_token_unix":1779779292.2507622,"t_done":348428.77325957,"t_done_unix":1779779293.7872028},{"request_id":"1268831:2:1271087:116","session_id":"1268831","input_length":19448,"t_proxy_recv":348424.75556342,"t_decision_unix":1779779289.7695048,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":12288,"affinity_cache_ratio":0.6318387494858083,"affinity_num_requests":1,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"cache_hit":12288,"estimated_new_tokens":7160,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":19448,"score_linear":135235.0,"score_lmetric":265334},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":19448,"score_linear":102074.0,"score_lmetric":117938},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":19448,"score_linear":126924.0,"score_lmetric":127428},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":108713,"ongoing_decode_tokens":15283,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":19448,"score_linear":108713.0,"score_lmetric":225756},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":55566,"ongoing_decode_tokens":27639,"pending_prefill_tokens":27927,"num_requests":1,"active_p_offloads":0,"cached_blocks":309,"cache_hit":12288,"new_prefill":7160,"score_linear":43278.0,"score_lmetric":35087},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":125063,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":19448,"score_linear":125063.0,"score_lmetric":38896},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":86709,"ongoing_decode_tokens":28906,"pending_prefill_tokens":7115,"num_requests":2,"active_p_offloads":0,"cached_blocks":342,"cache_hit":0,"new_prefill":19448,"score_linear":86709.0,"score_lmetric":53126},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":39513,"ongoing_decode_tokens":11128,"pending_prefill_tokens":10465,"num_requests":2,"active_p_offloads":0,"cached_blocks":288,"cache_hit":0,"new_prefill":19448,"score_linear":39513.0,"score_lmetric":59826}],"chosen_score_linear":43278.0,"chosen_score_lmetric":35087,"t_first_token":348426.423484739,"t_first_token_unix":1779779291.437428,"t_done":348428.943192789,"t_done_unix":1779779293.9571362},{"request_id":"1272085:1:1272085:125","session_id":"1272085","input_length":5302,"t_proxy_recv":348428.064087947,"t_decision_unix":1779779293.0780294,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.125,"fallback_score":10604,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":5302,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":5302,"score_linear":135235.0,"score_lmetric":237042},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":5302,"score_linear":102074.0,"score_lmetric":103792},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":5302,"score_linear":150574.0,"score_lmetric":246216},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":108713,"ongoing_decode_tokens":15283,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":5302,"score_linear":108713.0,"score_lmetric":197464},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":78261,"ongoing_decode_tokens":78261,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":363,"cache_hit":0,"new_prefill":5302,"score_linear":78261.0,"score_lmetric":15906},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":113923,"ongoing_decode_tokens":113923,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":269,"cache_hit":0,"new_prefill":5302,"score_linear":113923.0,"score_lmetric":10604},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":50480,"ongoing_decode_tokens":17706,"pending_prefill_tokens":4102,"num_requests":2,"active_p_offloads":0,"cached_blocks":411,"cache_hit":0,"new_prefill":5302,"score_linear":50480.0,"score_lmetric":18808},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":64508,"ongoing_decode_tokens":28385,"pending_prefill_tokens":21787,"num_requests":3,"active_p_offloads":0,"cached_blocks":309,"cache_hit":0,"new_prefill":5302,"score_linear":64508.0,"score_lmetric":81267}],"chosen_score_linear":113923.0,"chosen_score_lmetric":10604,"t_first_token":348428.521010098,"t_first_token_unix":1779779293.5349534,"t_done":348429.423869231,"t_done_unix":1779779294.4378128},{"request_id":"1270340:2:1271641:121","session_id":"1270340","input_length":28323,"t_proxy_recv":348426.934232065,"t_decision_unix":1779779291.9481738,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":27648,"affinity_cache_ratio":0.9761677788369876,"affinity_num_requests":1,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"cache_hit":27648,"estimated_new_tokens":675,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":28323,"score_linear":135235.0,"score_lmetric":283084},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":28323,"score_linear":102074.0,"score_lmetric":126813},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":28323,"score_linear":150574.0,"score_lmetric":292258},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":108713,"ongoing_decode_tokens":15283,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":28323,"score_linear":108713.0,"score_lmetric":243506},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":47087,"ongoing_decode_tokens":47087,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":363,"cache_hit":27648,"new_prefill":675,"score_linear":19439.0,"score_lmetric":675},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":113923,"ongoing_decode_tokens":113923,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":28323,"score_linear":113923.0,"score_lmetric":56646},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":46612,"ongoing_decode_tokens":28906,"pending_prefill_tokens":17706,"num_requests":2,"active_p_offloads":0,"cached_blocks":355,"cache_hit":0,"new_prefill":28323,"score_linear":46612.0,"score_lmetric":92058},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":64508,"ongoing_decode_tokens":28385,"pending_prefill_tokens":21787,"num_requests":3,"active_p_offloads":0,"cached_blocks":309,"cache_hit":0,"new_prefill":28323,"score_linear":64508.0,"score_lmetric":150330}],"chosen_score_linear":19439.0,"chosen_score_lmetric":675,"t_first_token":348427.109663947,"t_first_token_unix":1779779292.1236072,"t_done":348430.744740232,"t_done_unix":1779779295.758684},{"request_id":"1265942:3:1272453:127","session_id":"1265942","input_length":18093,"t_proxy_recv":348429.527785222,"t_decision_unix":1779779294.5417264,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":10240,"affinity_cache_ratio":0.5659647377438789,"affinity_num_requests":1,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"cache_hit":10240,"estimated_new_tokens":7853,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":18093,"score_linear":135235.0,"score_lmetric":262624},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":18093,"score_linear":102074.0,"score_lmetric":116583},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":18093,"score_linear":150574.0,"score_lmetric":271798},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":108713,"ongoing_decode_tokens":15283,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":18093,"score_linear":108713.0,"score_lmetric":223046},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":55962,"ongoing_decode_tokens":55962,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":382,"cache_hit":10240,"new_prefill":7853,"score_linear":45722.0,"score_lmetric":7853},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":113172,"ongoing_decode_tokens":113172,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":280,"cache_hit":0,"new_prefill":18093,"score_linear":113172.0,"score_lmetric":18093},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":32774,"ongoing_decode_tokens":32774,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":445,"cache_hit":0,"new_prefill":18093,"score_linear":32774.0,"score_lmetric":18093},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":74226,"ongoing_decode_tokens":74226,"pending_prefill_tokens":0,"num_requests":4,"active_p_offloads":0,"cached_blocks":309,"cache_hit":0,"new_prefill":18093,"score_linear":74226.0,"score_lmetric":72372}],"chosen_score_linear":45722.0,"chosen_score_lmetric":7853,"t_first_token":348430.521791107,"t_first_token_unix":1779779295.5357344,"t_done":348430.934892825,"t_done_unix":1779779295.9488358},{"request_id":"1271486:1:1271486:119","session_id":"1271486","input_length":21719,"t_proxy_recv":348426.001837667,"t_decision_unix":1779779291.015779,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.625,"fallback_score":21719,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":21719,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":21719,"score_linear":135235.0,"score_lmetric":269876},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":21719,"score_linear":102074.0,"score_lmetric":120209},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":21719,"score_linear":126924.0,"score_lmetric":129699},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":108713,"ongoing_decode_tokens":15283,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":21719,"score_linear":108713.0,"score_lmetric":230298},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":75014,"ongoing_decode_tokens":55566,"pending_prefill_tokens":7160,"num_requests":2,"active_p_offloads":0,"cached_blocks":309,"cache_hit":0,"new_prefill":21719,"score_linear":75014.0,"score_lmetric":57758},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":113923,"ongoing_decode_tokens":113172,"pending_prefill_tokens":751,"num_requests":2,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":21719,"score_linear":113923.0,"score_lmetric":44940},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":46612,"ongoing_decode_tokens":28906,"pending_prefill_tokens":17706,"num_requests":2,"active_p_offloads":0,"cached_blocks":355,"cache_hit":0,"new_prefill":21719,"score_linear":46612.0,"score_lmetric":78850},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":28385,"ongoing_decode_tokens":28385,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":309,"cache_hit":0,"new_prefill":21719,"score_linear":28385.0,"score_lmetric":21719}],"chosen_score_linear":28385.0,"chosen_score_lmetric":21719,"t_first_token":348428.364567095,"t_first_token_unix":1779779293.3785102,"t_done":348431.507722012,"t_done_unix":1779779296.5216653},{"request_id":"1266668:2:1270661:115","session_id":"1266668","input_length":28385,"t_proxy_recv":348423.123149839,"t_decision_unix":1779779288.1370914,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":17920,"affinity_cache_ratio":0.6313193588162762,"affinity_num_requests":1,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":17920,"estimated_new_tokens":10465,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":4096,"new_prefill":24289,"score_linear":131139.0,"score_lmetric":275016},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":28385,"score_linear":102074.0,"score_lmetric":126875},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":28385,"score_linear":126924.0,"score_lmetric":136365},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":108713,"ongoing_decode_tokens":15283,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":28385,"score_linear":108713.0,"score_lmetric":243630},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":55566,"ongoing_decode_tokens":27639,"pending_prefill_tokens":27927,"num_requests":1,"active_p_offloads":0,"cached_blocks":309,"cache_hit":0,"new_prefill":28385,"score_linear":55566.0,"score_lmetric":56312},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":28385,"score_linear":125063.0,"score_lmetric":291536},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":86709,"ongoing_decode_tokens":28906,"pending_prefill_tokens":7115,"num_requests":2,"active_p_offloads":0,"cached_blocks":342,"cache_hit":0,"new_prefill":28385,"score_linear":86709.0,"score_lmetric":71000},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":11128,"ongoing_decode_tokens":0,"pending_prefill_tokens":11128,"num_requests":1,"active_p_offloads":0,"cached_blocks":288,"cache_hit":17920,"new_prefill":10465,"score_linear":-6792.0,"score_lmetric":21593}],"chosen_score_linear":-6792.0,"chosen_score_lmetric":21593,"t_first_token":348425.294849644,"t_first_token_unix":1779779290.3087933,"t_done":348431.758275946,"t_done_unix":1779779296.7722192},{"request_id":"1262354:6:1270643:114","session_id":"1262354","input_length":32774,"t_proxy_recv":348427.268676023,"t_decision_unix":1779779292.2826176,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":28672,"affinity_cache_ratio":0.8748398120461341,"affinity_num_requests":1,"avg_num_requests":2.0,"fallback_score":null,"tie_break_used":false,"cache_hit":28672,"estimated_new_tokens":4102,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":32774,"score_linear":135235.0,"score_lmetric":291986},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":32774,"score_linear":102074.0,"score_lmetric":131264},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":32774,"score_linear":150574.0,"score_lmetric":301160},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":108713,"ongoing_decode_tokens":15283,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":32774,"score_linear":108713.0,"score_lmetric":252408},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":75410,"ongoing_decode_tokens":75410,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":363,"cache_hit":0,"new_prefill":32774,"score_linear":75410.0,"score_lmetric":65548},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":116447,"ongoing_decode_tokens":113923,"pending_prefill_tokens":2524,"num_requests":3,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":32774,"score_linear":116447.0,"score_lmetric":105894},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":17706,"ongoing_decode_tokens":17706,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":411,"cache_hit":28672,"new_prefill":4102,"score_linear":-10966.0,"score_lmetric":4102},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":64508,"ongoing_decode_tokens":28385,"pending_prefill_tokens":21787,"num_requests":3,"active_p_offloads":0,"cached_blocks":309,"cache_hit":28672,"new_prefill":4102,"score_linear":35836.0,"score_lmetric":77667}],"chosen_score_linear":-10966.0,"chosen_score_lmetric":4102,"t_first_token":348428.188802174,"t_first_token_unix":1779779293.2027457,"t_done":348433.193036047,"t_done_unix":1779779298.2069798},{"request_id":"1258859:2:1264639:81","session_id":"1258859","input_length":113172,"t_proxy_recv":348424.945562452,"t_decision_unix":1779779289.9595041,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":112640,"affinity_cache_ratio":0.995299190612519,"affinity_num_requests":0,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":112640,"estimated_new_tokens":532,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":113172,"score_linear":135235.0,"score_lmetric":452782},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":113172,"score_linear":102074.0,"score_lmetric":211662},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":113172,"score_linear":126924.0,"score_lmetric":221152},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":108713,"ongoing_decode_tokens":15283,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":113172,"score_linear":108713.0,"score_lmetric":413204},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":75014,"ongoing_decode_tokens":27639,"pending_prefill_tokens":35087,"num_requests":2,"active_p_offloads":0,"cached_blocks":309,"cache_hit":0,"new_prefill":113172,"score_linear":75014.0,"score_lmetric":296518},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":265,"cache_hit":112640,"new_prefill":532,"score_linear":-112640.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":86709,"ongoing_decode_tokens":86709,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":342,"cache_hit":0,"new_prefill":113172,"score_linear":86709.0,"score_lmetric":226344},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":39513,"ongoing_decode_tokens":11128,"pending_prefill_tokens":10465,"num_requests":2,"active_p_offloads":0,"cached_blocks":288,"cache_hit":0,"new_prefill":113172,"score_linear":39513.0,"score_lmetric":247274}],"chosen_score_linear":-112640.0,"chosen_score_lmetric":0,"t_first_token":348425.295732685,"t_first_token_unix":1779779290.3096774,"t_done":348434.445136379,"t_done_unix":1779779299.4590802},{"request_id":"1272085:2:1273645:130","session_id":"1272085","input_length":8017,"t_proxy_recv":348433.696765518,"t_decision_unix":1779779298.7107067,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":5120,"affinity_cache_ratio":0.6386428838717725,"affinity_num_requests":1,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":5120,"estimated_new_tokens":2897,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":8017,"score_linear":135235.0,"score_lmetric":242472},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":8017,"score_linear":102074.0,"score_lmetric":106507},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":8017,"score_linear":150574.0,"score_lmetric":251646},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":108713,"ongoing_decode_tokens":15283,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":8017,"score_linear":108713.0,"score_lmetric":202894},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":49602,"ongoing_decode_tokens":27639,"pending_prefill_tokens":21963,"num_requests":1,"active_p_offloads":0,"cached_blocks":398,"cache_hit":0,"new_prefill":8017,"score_linear":49602.0,"score_lmetric":29980},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":113172,"ongoing_decode_tokens":113172,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":280,"cache_hit":5120,"new_prefill":2897,"score_linear":108052.0,"score_lmetric":2897},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":454,"cache_hit":0,"new_prefill":8017,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":57833,"ongoing_decode_tokens":57833,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":372,"cache_hit":0,"new_prefill":8017,"score_linear":57833.0,"score_lmetric":24051}],"chosen_score_linear":108052.0,"chosen_score_lmetric":2897,"t_first_token":348434.081466819,"t_first_token_unix":1779779299.09541,"t_done":348434.638239722,"t_done_unix":1779779299.6521828},{"request_id":"1272313:1:1272313:126","session_id":"1272313","input_length":9718,"t_proxy_recv":348428.976846748,"t_decision_unix":1779779293.990788,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.75,"fallback_score":6114,"tie_break_used":false,"cache_hit":7680,"estimated_new_tokens":2038,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":4096,"new_prefill":5622,"score_linear":131139.0,"score_lmetric":237682},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":9718,"score_linear":102074.0,"score_lmetric":108208},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":9718,"score_linear":150574.0,"score_lmetric":255048},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":108713,"ongoing_decode_tokens":15283,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":9718,"score_linear":108713.0,"score_lmetric":206296},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":55962,"ongoing_decode_tokens":55962,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":382,"cache_hit":0,"new_prefill":9718,"score_linear":55962.0,"score_lmetric":9718},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":118474,"ongoing_decode_tokens":118474,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":270,"cache_hit":0,"new_prefill":9718,"score_linear":118474.0,"score_lmetric":19436},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":32774,"ongoing_decode_tokens":32774,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":445,"cache_hit":0,"new_prefill":9718,"score_linear":32774.0,"score_lmetric":9718},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":64508,"ongoing_decode_tokens":64508,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":309,"cache_hit":7680,"new_prefill":2038,"score_linear":56828.0,"score_lmetric":6114}],"chosen_score_linear":56828.0,"chosen_score_lmetric":6114,"t_first_token":348429.216766113,"t_first_token_unix":1779779294.2307093,"t_done":348435.241700258,"t_done_unix":1779779300.2556434},{"request_id":"1269373:2:1271580:120","session_id":"1269373","input_length":14404,"t_proxy_recv":348426.312622902,"t_decision_unix":1779779291.3265646,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":14336,"affinity_cache_ratio":0.995279089141905,"affinity_num_requests":2,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"cache_hit":14336,"estimated_new_tokens":68,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":4096,"new_prefill":10308,"score_linear":131139.0,"score_lmetric":247054},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":14404,"score_linear":102074.0,"score_lmetric":112894},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":14404,"score_linear":126924.0,"score_lmetric":122384},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":108713,"ongoing_decode_tokens":15283,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":14404,"score_linear":108713.0,"score_lmetric":215668},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":75014,"ongoing_decode_tokens":55566,"pending_prefill_tokens":7160,"num_requests":2,"active_p_offloads":0,"cached_blocks":309,"cache_hit":0,"new_prefill":14404,"score_linear":75014.0,"score_lmetric":43128},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":113923,"ongoing_decode_tokens":113923,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":14404,"score_linear":113923.0,"score_lmetric":28808},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":46612,"ongoing_decode_tokens":28906,"pending_prefill_tokens":17706,"num_requests":2,"active_p_offloads":0,"cached_blocks":355,"cache_hit":0,"new_prefill":14404,"score_linear":46612.0,"score_lmetric":64220},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":50104,"ongoing_decode_tokens":28385,"pending_prefill_tokens":21719,"num_requests":2,"active_p_offloads":0,"cached_blocks":309,"cache_hit":14336,"new_prefill":68,"score_linear":35768.0,"score_lmetric":43574}],"chosen_score_linear":35768.0,"chosen_score_lmetric":43574,"t_first_token":348428.365135463,"t_first_token_unix":1779779293.3790784,"t_done":348435.428246018,"t_done_unix":1779779300.442189},{"request_id":"1260327:3:1273741:132","session_id":"1260327","input_length":31283,"t_proxy_recv":348434.103033982,"t_decision_unix":1779779299.1169753,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":30720,"affinity_cache_ratio":0.9820030048269028,"affinity_num_requests":3,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"cache_hit":30720,"estimated_new_tokens":563,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":31283,"score_linear":135235.0,"score_lmetric":289004},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":31283,"score_linear":102074.0,"score_lmetric":129773},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":31283,"score_linear":150574.0,"score_lmetric":298178},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":108713,"ongoing_decode_tokens":15283,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":31283,"score_linear":108713.0,"score_lmetric":249426},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":49602,"ongoing_decode_tokens":27639,"pending_prefill_tokens":21963,"num_requests":1,"active_p_offloads":0,"cached_blocks":398,"cache_hit":0,"new_prefill":31283,"score_linear":49602.0,"score_lmetric":53246},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":121189,"ongoing_decode_tokens":121189,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":280,"cache_hit":0,"new_prefill":31283,"score_linear":121189.0,"score_lmetric":62566},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":25108,"ongoing_decode_tokens":0,"pending_prefill_tokens":7700,"num_requests":1,"active_p_offloads":0,"cached_blocks":454,"cache_hit":0,"new_prefill":31283,"score_linear":25108.0,"score_lmetric":38983},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":57833,"ongoing_decode_tokens":57833,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":372,"cache_hit":30720,"new_prefill":563,"score_linear":27113.0,"score_lmetric":1689}],"chosen_score_linear":27113.0,"chosen_score_lmetric":1689,"t_first_token":348434.293060682,"t_first_token_unix":1779779299.3070042,"t_done":348435.455740912,"t_done_unix":1779779300.4696844},{"request_id":"1266668:3:1273119:128","session_id":"1266668","input_length":33711,"t_proxy_recv":348431.76925139,"t_decision_unix":1779779296.783193,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":28672,"affinity_cache_ratio":0.850523567974845,"affinity_num_requests":2,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":28672,"estimated_new_tokens":5039,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":4096,"new_prefill":29615,"score_linear":131139.0,"score_lmetric":285668},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":33711,"score_linear":102074.0,"score_lmetric":132201},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":33711,"score_linear":150574.0,"score_lmetric":303034},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":108713,"ongoing_decode_tokens":15283,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":33711,"score_linear":108713.0,"score_lmetric":254282},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":398,"cache_hit":0,"new_prefill":33711,"score_linear":27639.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":113172,"ongoing_decode_tokens":113172,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":280,"cache_hit":0,"new_prefill":33711,"score_linear":113172.0,"score_lmetric":33711},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":32774,"ongoing_decode_tokens":32774,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":445,"cache_hit":0,"new_prefill":33711,"score_linear":32774.0,"score_lmetric":33711},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":24122,"ongoing_decode_tokens":24122,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":372,"cache_hit":28672,"new_prefill":5039,"score_linear":-4550.0,"score_lmetric":10078}],"chosen_score_linear":-4550.0,"chosen_score_lmetric":10078,"t_first_token":348432.903935467,"t_first_token_unix":1779779297.9178793,"t_done":348436.546218298,"t_done_unix":1779779301.5601616},{"request_id":"1265942:4:1274182:133","session_id":"1265942","input_length":21229,"t_proxy_recv":348435.578419328,"t_decision_unix":1779779300.592361,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":17920,"affinity_cache_ratio":0.8441283150407461,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":17920,"estimated_new_tokens":3309,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":21229,"score_linear":135235.0,"score_lmetric":268896},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":21229,"score_linear":102074.0,"score_lmetric":119719},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":21229,"score_linear":150574.0,"score_lmetric":278070},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":108713,"ongoing_decode_tokens":15283,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":21229,"score_linear":108713.0,"score_lmetric":229318},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":49602,"ongoing_decode_tokens":49602,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":398,"cache_hit":17920,"new_prefill":3309,"score_linear":31682.0,"score_lmetric":3309},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":287,"cache_hit":0,"new_prefill":21229,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":25108,"ongoing_decode_tokens":25108,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":454,"cache_hit":0,"new_prefill":21229,"score_linear":25108.0,"score_lmetric":21229},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":33711,"ongoing_decode_tokens":33711,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":378,"cache_hit":0,"new_prefill":21229,"score_linear":33711.0,"score_lmetric":21229}],"chosen_score_linear":31682.0,"chosen_score_lmetric":3309,"t_first_token":348436.130821917,"t_first_token_unix":1779779301.1447651,"t_done":348437.17738087,"t_done_unix":1779779302.1913242},{"request_id":"1274405:1:1274405:134","session_id":"1274405","input_length":6208,"t_proxy_recv":348436.494518949,"t_decision_unix":1779779301.50846,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":6208,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":6208,"score_linear":135235.0,"score_lmetric":238854},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":6208,"score_linear":102074.0,"score_lmetric":104698},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":6208,"score_linear":150574.0,"score_lmetric":248028},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":108713,"ongoing_decode_tokens":15283,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":6208,"score_linear":108713.0,"score_lmetric":199276},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":70831,"ongoing_decode_tokens":70831,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":398,"cache_hit":0,"new_prefill":6208,"score_linear":70831.0,"score_lmetric":12416},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":287,"cache_hit":0,"new_prefill":6208,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":25108,"ongoing_decode_tokens":25108,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":454,"cache_hit":0,"new_prefill":6208,"score_linear":25108.0,"score_lmetric":6208},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":33711,"ongoing_decode_tokens":33711,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":378,"cache_hit":0,"new_prefill":6208,"score_linear":33711.0,"score_lmetric":6208}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348436.902861569,"t_first_token_unix":1779779301.9168048,"t_done":348437.381480123,"t_done_unix":1779779302.3954244},{"request_id":"1271328:2:1273711:131","session_id":"1271328","input_length":25108,"t_proxy_recv":348433.958020625,"t_decision_unix":1779779298.9719622,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":17408,"affinity_cache_ratio":0.6933248367054325,"affinity_num_requests":0,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"cache_hit":17408,"estimated_new_tokens":7700,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":25108,"score_linear":135235.0,"score_lmetric":276654},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":25108,"score_linear":102074.0,"score_lmetric":123598},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":25108,"score_linear":150574.0,"score_lmetric":285828},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":108713,"ongoing_decode_tokens":15283,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":25108,"score_linear":108713.0,"score_lmetric":237076},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":49602,"ongoing_decode_tokens":27639,"pending_prefill_tokens":21963,"num_requests":1,"active_p_offloads":0,"cached_blocks":398,"cache_hit":0,"new_prefill":25108,"score_linear":49602.0,"score_lmetric":47071},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":121189,"ongoing_decode_tokens":113172,"pending_prefill_tokens":2897,"num_requests":2,"active_p_offloads":0,"cached_blocks":280,"cache_hit":0,"new_prefill":25108,"score_linear":121189.0,"score_lmetric":56010},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":454,"cache_hit":17408,"new_prefill":7700,"score_linear":-17408.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":57833,"ongoing_decode_tokens":57833,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":372,"cache_hit":0,"new_prefill":25108,"score_linear":57833.0,"score_lmetric":75324}],"chosen_score_linear":-17408.0,"chosen_score_lmetric":0,"t_first_token":348435.213084446,"t_first_token_unix":1779779300.227028,"t_done":348438.017774663,"t_done_unix":1779779303.0317185},{"request_id":"1274890:1:1274890:135","session_id":"1274890","input_length":10506,"t_proxy_recv":348438.020768762,"t_decision_unix":1779779303.0347102,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":10506,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":10506,"score_linear":135235.0,"score_lmetric":247450},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":10506,"score_linear":102074.0,"score_lmetric":108996},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":10506,"score_linear":150574.0,"score_lmetric":256624},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":108713,"ongoing_decode_tokens":15283,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":10506,"score_linear":108713.0,"score_lmetric":207872},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":49602,"ongoing_decode_tokens":49602,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":404,"cache_hit":0,"new_prefill":10506,"score_linear":49602.0,"score_lmetric":10506},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":299,"cache_hit":0,"new_prefill":10506,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":469,"cache_hit":0,"new_prefill":10506,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":388,"cache_hit":0,"new_prefill":10506,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348438.843122768,"t_first_token_unix":1779779303.8570664,"t_done":348439.093787072,"t_done_unix":1779779304.1077306},{"request_id":"1273203:1:1273203:129","session_id":"1273203","input_length":21963,"t_proxy_recv":348432.053933364,"t_decision_unix":1779779297.0678747,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.5,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":21963,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":21963,"score_linear":135235.0,"score_lmetric":270364},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":21963,"score_linear":102074.0,"score_lmetric":120453},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":21963,"score_linear":150574.0,"score_lmetric":279538},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":108713,"ongoing_decode_tokens":15283,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":21963,"score_linear":108713.0,"score_lmetric":230786},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":398,"cache_hit":0,"new_prefill":21963,"score_linear":27639.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":113172,"ongoing_decode_tokens":113172,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":280,"cache_hit":0,"new_prefill":21963,"score_linear":113172.0,"score_lmetric":21963},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":32774,"ongoing_decode_tokens":32774,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":445,"cache_hit":0,"new_prefill":21963,"score_linear":32774.0,"score_lmetric":21963},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":57833,"ongoing_decode_tokens":24122,"pending_prefill_tokens":5039,"num_requests":3,"active_p_offloads":0,"cached_blocks":372,"cache_hit":12800,"new_prefill":9163,"score_linear":45033.0,"score_lmetric":42606}],"chosen_score_linear":27639.0,"chosen_score_lmetric":0,"t_first_token":348434.434753168,"t_first_token_unix":1779779299.448697,"t_done":348440.225647455,"t_done_unix":1779779305.2395911},{"request_id":"1275433:1:1275433:137","session_id":"1275433","input_length":7794,"t_proxy_recv":348439.770114352,"t_decision_unix":1779779304.7840538,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":7794,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":7794,"score_linear":135235.0,"score_lmetric":242026},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":7794,"score_linear":102074.0,"score_lmetric":106284},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":7794,"score_linear":150574.0,"score_lmetric":251200},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":108713,"ongoing_decode_tokens":15283,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":7794,"score_linear":108713.0,"score_lmetric":202448},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":49602,"ongoing_decode_tokens":49602,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":404,"cache_hit":0,"new_prefill":7794,"score_linear":49602.0,"score_lmetric":7794},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":299,"cache_hit":0,"new_prefill":7794,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":489,"cache_hit":0,"new_prefill":7794,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":114612,"ongoing_decode_tokens":0,"pending_prefill_tokens":114612,"num_requests":1,"active_p_offloads":0,"cached_blocks":388,"cache_hit":0,"new_prefill":7794,"score_linear":114612.0,"score_lmetric":122406}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348440.310498031,"t_first_token_unix":1779779305.3244421,"t_done":348440.362651133,"t_done_unix":1779779305.3765948},{"request_id":"1275433:2:1276027:139","session_id":"1275433","input_length":7463,"t_proxy_recv":348442.04209997,"t_decision_unix":1779779307.0560408,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":7168,"affinity_cache_ratio":0.960471660190272,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":7168,"estimated_new_tokens":295,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":7463,"score_linear":135235.0,"score_lmetric":241364},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":7463,"score_linear":102074.0,"score_lmetric":105953},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":7463,"score_linear":150574.0,"score_lmetric":250538},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":108713,"ongoing_decode_tokens":15283,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":7463,"score_linear":108713.0,"score_lmetric":201786},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":447,"cache_hit":0,"new_prefill":7463,"score_linear":27639.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":314,"cache_hit":7168,"new_prefill":295,"score_linear":-7168.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":489,"cache_hit":0,"new_prefill":7463,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":125251,"ongoing_decode_tokens":0,"pending_prefill_tokens":115523,"num_requests":2,"active_p_offloads":0,"cached_blocks":388,"cache_hit":0,"new_prefill":7463,"score_linear":125251.0,"score_lmetric":245972}],"chosen_score_linear":-7168.0,"chosen_score_lmetric":0,"t_first_token":348442.067544302,"t_first_token_unix":1779779307.081488,"t_done":348442.076971908,"t_done_unix":1779779307.090915},{"request_id":"1264679:1:1264679:82","session_id":"1264679","input_length":110048,"t_proxy_recv":348402.662106694,"t_decision_unix":1779779267.6760478,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":110048,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":110048,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55989,"ongoing_decode_tokens":55989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":261,"cache_hit":0,"new_prefill":110048,"score_linear":55989.0,"score_lmetric":110048},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":296,"cache_hit":0,"new_prefill":110048,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":71575,"ongoing_decode_tokens":15283,"pending_prefill_tokens":56292,"num_requests":2,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":110048,"score_linear":71575.0,"score_lmetric":332680},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":113053,"ongoing_decode_tokens":0,"pending_prefill_tokens":92573,"num_requests":1,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":110048,"score_linear":113053.0,"score_lmetric":202621},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":110048,"score_linear":125063.0,"score_lmetric":454862},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":50783,"ongoing_decode_tokens":0,"pending_prefill_tokens":50783,"num_requests":1,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":110048,"score_linear":50783.0,"score_lmetric":160831},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":141,"cache_hit":0,"new_prefill":110048,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348442.409241101,"t_first_token_unix":1779779307.4231846,"t_done":348443.512073565,"t_done_unix":1779779308.5260174},{"request_id":"1276111:1:1276111:140","session_id":"1276111","input_length":15783,"t_proxy_recv":348442.390848034,"t_decision_unix":1779779307.4047892,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":15783,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":15783,"score_linear":135235.0,"score_lmetric":258004},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":15783,"score_linear":102074.0,"score_lmetric":114273},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":15783,"score_linear":150574.0,"score_lmetric":267178},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":108713,"ongoing_decode_tokens":15283,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":15783,"score_linear":108713.0,"score_lmetric":218426},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":447,"cache_hit":0,"new_prefill":15783,"score_linear":27639.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":314,"cache_hit":0,"new_prefill":15783,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":489,"cache_hit":0,"new_prefill":15783,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":125251,"ongoing_decode_tokens":0,"pending_prefill_tokens":115523,"num_requests":2,"active_p_offloads":0,"cached_blocks":388,"cache_hit":0,"new_prefill":15783,"score_linear":125251.0,"score_lmetric":262612}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348443.824948457,"t_first_token_unix":1779779308.838892,"t_done":348444.012357399,"t_done_unix":1779779309.0263007},{"request_id":"1276498:1:1276498:142","session_id":"1276498","input_length":8704,"t_proxy_recv":348443.722653951,"t_decision_unix":1779779308.7365952,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":8704,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":371,"cache_hit":0,"new_prefill":8704,"score_linear":25187.0,"score_lmetric":8704},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":8704,"score_linear":102074.0,"score_lmetric":107194},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":8704,"score_linear":150574.0,"score_lmetric":253020},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":108713,"ongoing_decode_tokens":15283,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":8704,"score_linear":108713.0,"score_lmetric":204268},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":44919,"ongoing_decode_tokens":27639,"pending_prefill_tokens":17280,"num_requests":1,"active_p_offloads":0,"cached_blocks":447,"cache_hit":0,"new_prefill":8704,"score_linear":44919.0,"score_lmetric":25984},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":15783,"ongoing_decode_tokens":0,"pending_prefill_tokens":15783,"num_requests":1,"active_p_offloads":0,"cached_blocks":314,"cache_hit":0,"new_prefill":8704,"score_linear":15783.0,"score_lmetric":24487},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":489,"cache_hit":0,"new_prefill":8704,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":125251,"ongoing_decode_tokens":0,"pending_prefill_tokens":115523,"num_requests":2,"active_p_offloads":0,"cached_blocks":388,"cache_hit":0,"new_prefill":8704,"score_linear":125251.0,"score_lmetric":248454}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348444.367584051,"t_first_token_unix":1779779309.3815272,"t_done":348444.625245425,"t_done_unix":1779779309.6391885},{"request_id":"1274405:2:1276717:144","session_id":"1274405","input_length":6360,"t_proxy_recv":348444.505151337,"t_decision_unix":1779779309.5190926,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":6144,"affinity_cache_ratio":0.9660377358490566,"affinity_num_requests":0,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":6144,"estimated_new_tokens":216,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":371,"cache_hit":0,"new_prefill":6360,"score_linear":25187.0,"score_lmetric":6360},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":6360,"score_linear":102074.0,"score_lmetric":104850},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":6360,"score_linear":150574.0,"score_lmetric":248332},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":108713,"ongoing_decode_tokens":15283,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":6360,"score_linear":108713.0,"score_lmetric":199580},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":44919,"ongoing_decode_tokens":44919,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":447,"cache_hit":0,"new_prefill":6360,"score_linear":44919.0,"score_lmetric":6360},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":344,"cache_hit":6144,"new_prefill":216,"score_linear":-6144.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":8704,"ongoing_decode_tokens":8704,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":489,"cache_hit":0,"new_prefill":6360,"score_linear":8704.0,"score_lmetric":6360},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":174631,"ongoing_decode_tokens":0,"pending_prefill_tokens":131111,"num_requests":3,"active_p_offloads":0,"cached_blocks":388,"cache_hit":0,"new_prefill":6360,"score_linear":174631.0,"score_lmetric":412413}],"chosen_score_linear":-6144.0,"chosen_score_lmetric":0,"t_first_token":348444.540433493,"t_first_token_unix":1779779309.5543773,"t_done":348444.710254441,"t_done_unix":1779779309.7241983},{"request_id":"1276136:1:1276136:141","session_id":"1276136","input_length":17280,"t_proxy_recv":348442.484594489,"t_decision_unix":1779779307.4985356,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":17280,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":135235,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":17280,"score_linear":135235.0,"score_lmetric":34560},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":17280,"score_linear":102074.0,"score_lmetric":115770},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":17280,"score_linear":150574.0,"score_lmetric":270172},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":108713,"ongoing_decode_tokens":15283,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":17280,"score_linear":108713.0,"score_lmetric":221420},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":447,"cache_hit":0,"new_prefill":17280,"score_linear":27639.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":15783,"ongoing_decode_tokens":0,"pending_prefill_tokens":15783,"num_requests":1,"active_p_offloads":0,"cached_blocks":314,"cache_hit":0,"new_prefill":17280,"score_linear":15783.0,"score_lmetric":33063},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":489,"cache_hit":0,"new_prefill":17280,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":125251,"ongoing_decode_tokens":0,"pending_prefill_tokens":115523,"num_requests":2,"active_p_offloads":0,"cached_blocks":388,"cache_hit":0,"new_prefill":17280,"score_linear":125251.0,"score_lmetric":265606}],"chosen_score_linear":27639.0,"chosen_score_lmetric":0,"t_first_token":348444.184976871,"t_first_token_unix":1779779309.1989202,"t_done":348444.739653147,"t_done_unix":1779779309.7535963},{"request_id":"1277533:1:1277533:150","session_id":"1277533","input_length":4729,"t_proxy_recv":348447.523291797,"t_decision_unix":1779779312.537233,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.75,"fallback_score":4729,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":4729,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":371,"cache_hit":0,"new_prefill":4729,"score_linear":25187.0,"score_lmetric":4729},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":4729,"score_linear":102074.0,"score_lmetric":103219},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":4729,"score_linear":150574.0,"score_lmetric":245070},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":108713,"ongoing_decode_tokens":15283,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":4729,"score_linear":108713.0,"score_lmetric":196318},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":70776,"ongoing_decode_tokens":38931,"pending_prefill_tokens":12389,"num_requests":2,"active_p_offloads":0,"cached_blocks":480,"cache_hit":0,"new_prefill":4729,"score_linear":70776.0,"score_lmetric":34236},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":27221,"ongoing_decode_tokens":0,"pending_prefill_tokens":27221,"num_requests":1,"active_p_offloads":0,"cached_blocks":344,"cache_hit":0,"new_prefill":4729,"score_linear":27221.0,"score_lmetric":31950},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":33524,"ongoing_decode_tokens":33524,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":506,"cache_hit":0,"new_prefill":4729,"score_linear":33524.0,"score_lmetric":4729},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":388,"cache_hit":0,"new_prefill":4729,"score_linear":191531.0,"score_lmetric":551568}],"chosen_score_linear":33524.0,"chosen_score_lmetric":4729,"t_first_token":348447.830255589,"t_first_token_unix":1779779312.8441987,"t_done":348448.047853841,"t_done_unix":1779779313.0617971},{"request_id":"1262354:7:1277104:146","session_id":"1262354","input_length":33524,"t_proxy_recv":348445.899307796,"t_decision_unix":1779779310.9132493,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":33280,"affinity_cache_ratio":0.992721632263453,"affinity_num_requests":0,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":33280,"estimated_new_tokens":244,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":371,"cache_hit":0,"new_prefill":33524,"score_linear":25187.0,"score_lmetric":33524},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":33524,"score_linear":102074.0,"score_lmetric":132014},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":33524,"score_linear":150574.0,"score_lmetric":302660},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":108713,"ongoing_decode_tokens":15283,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":33524,"score_linear":108713.0,"score_lmetric":253908},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":38931,"ongoing_decode_tokens":27639,"pending_prefill_tokens":8732,"num_requests":1,"active_p_offloads":0,"cached_blocks":480,"cache_hit":0,"new_prefill":33524,"score_linear":38931.0,"score_lmetric":42256},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":344,"cache_hit":0,"new_prefill":33524,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":506,"cache_hit":33280,"new_prefill":244,"score_linear":-33280.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":174631,"ongoing_decode_tokens":0,"pending_prefill_tokens":131111,"num_requests":3,"active_p_offloads":0,"cached_blocks":388,"cache_hit":28672,"new_prefill":4852,"score_linear":145959.0,"score_lmetric":407889}],"chosen_score_linear":-33280.0,"chosen_score_lmetric":0,"t_first_token":348445.969039644,"t_first_token_unix":1779779310.9829826,"t_done":348448.79486182,"t_done_unix":1779779313.8088057},{"request_id":"1271991:2:1276999:145","session_id":"1271991","input_length":11292,"t_proxy_recv":348445.49202913,"t_decision_unix":1779779310.5059705,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":2560,"affinity_cache_ratio":0.2267091746369111,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":false,"cache_hit":2560,"estimated_new_tokens":8732,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":371,"cache_hit":0,"new_prefill":11292,"score_linear":25187.0,"score_lmetric":11292},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":11292,"score_linear":102074.0,"score_lmetric":109782},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":11292,"score_linear":150574.0,"score_lmetric":258196},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":108713,"ongoing_decode_tokens":15283,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":11292,"score_linear":108713.0,"score_lmetric":209444},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":480,"cache_hit":2560,"new_prefill":8732,"score_linear":25079.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":344,"cache_hit":0,"new_prefill":11292,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":506,"cache_hit":0,"new_prefill":11292,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":174631,"ongoing_decode_tokens":0,"pending_prefill_tokens":131111,"num_requests":3,"active_p_offloads":0,"cached_blocks":388,"cache_hit":0,"new_prefill":11292,"score_linear":174631.0,"score_lmetric":427209}],"chosen_score_linear":25079.0,"chosen_score_lmetric":0,"t_first_token":348446.25059437,"t_first_token_unix":1779779311.2645376,"t_done":348449.025581853,"t_done_unix":1779779314.0395248},{"request_id":"1268831:3:1277112:147","session_id":"1268831","input_length":31845,"t_proxy_recv":348445.938393283,"t_decision_unix":1779779310.9523346,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":19456,"affinity_cache_ratio":0.6109593342753965,"affinity_num_requests":1,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":19456,"estimated_new_tokens":12389,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":371,"cache_hit":0,"new_prefill":31845,"score_linear":25187.0,"score_lmetric":31845},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":31845,"score_linear":102074.0,"score_lmetric":130335},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":31845,"score_linear":150574.0,"score_lmetric":299302},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":108713,"ongoing_decode_tokens":15283,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":31845,"score_linear":108713.0,"score_lmetric":250550},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":38931,"ongoing_decode_tokens":27639,"pending_prefill_tokens":8732,"num_requests":1,"active_p_offloads":0,"cached_blocks":480,"cache_hit":19456,"new_prefill":12389,"score_linear":19475.0,"score_lmetric":21121},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":344,"cache_hit":0,"new_prefill":31845,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":33524,"ongoing_decode_tokens":0,"pending_prefill_tokens":244,"num_requests":1,"active_p_offloads":0,"cached_blocks":506,"cache_hit":0,"new_prefill":31845,"score_linear":33524.0,"score_lmetric":32089},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":174631,"ongoing_decode_tokens":0,"pending_prefill_tokens":131111,"num_requests":3,"active_p_offloads":0,"cached_blocks":388,"cache_hit":0,"new_prefill":31845,"score_linear":174631.0,"score_lmetric":488868}],"chosen_score_linear":19475.0,"chosen_score_lmetric":21121,"t_first_token":348448.600448379,"t_first_token_unix":1779779313.6143923,"t_done":348451.242700411,"t_done_unix":1779779316.2566433},{"request_id":"1273203:2:1277744:152","session_id":"1273203","input_length":22573,"t_proxy_recv":348448.247993733,"t_decision_unix":1779779313.261935,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":22016,"affinity_cache_ratio":0.9753245027244939,"affinity_num_requests":2,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"cache_hit":22016,"estimated_new_tokens":557,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":34071,"ongoing_decode_tokens":34071,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":371,"cache_hit":0,"new_prefill":22573,"score_linear":34071.0,"score_lmetric":45146},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":22573,"score_linear":102074.0,"score_lmetric":121063},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":22573,"score_linear":150574.0,"score_lmetric":280758},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":108713,"ongoing_decode_tokens":15283,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":22573,"score_linear":108713.0,"score_lmetric":232006},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":70776,"ongoing_decode_tokens":38931,"pending_prefill_tokens":12389,"num_requests":2,"active_p_offloads":0,"cached_blocks":480,"cache_hit":22016,"new_prefill":557,"score_linear":48760.0,"score_lmetric":25892},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":27221,"ongoing_decode_tokens":0,"pending_prefill_tokens":27221,"num_requests":1,"active_p_offloads":0,"cached_blocks":344,"cache_hit":0,"new_prefill":22573,"score_linear":27221.0,"score_lmetric":49794},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":33524,"ongoing_decode_tokens":33524,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":515,"cache_hit":0,"new_prefill":22573,"score_linear":33524.0,"score_lmetric":22573},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":388,"cache_hit":12800,"new_prefill":9773,"score_linear":178731.0,"score_lmetric":571744}],"chosen_score_linear":48760.0,"chosen_score_lmetric":25892,"t_first_token":348448.655023485,"t_first_token_unix":1779779313.668967,"t_done":348451.553968355,"t_done_unix":1779779316.5679114},{"request_id":"1271459:2:1277544:151","session_id":"1271459","input_length":8884,"t_proxy_recv":348447.560934583,"t_decision_unix":1779779312.574876,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":5,"chosen_idx":0,"affinity_cache_hit":512,"affinity_cache_ratio":0.05763169743358847,"affinity_num_requests":1,"avg_num_requests":1.875,"fallback_score":8884,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":8884,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":371,"cache_hit":0,"new_prefill":8884,"score_linear":25187.0,"score_lmetric":8884},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":8884,"score_linear":102074.0,"score_lmetric":107374},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":8884,"score_linear":150574.0,"score_lmetric":253380},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":108713,"ongoing_decode_tokens":15283,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":8884,"score_linear":108713.0,"score_lmetric":204628},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":70776,"ongoing_decode_tokens":38931,"pending_prefill_tokens":12389,"num_requests":2,"active_p_offloads":0,"cached_blocks":480,"cache_hit":0,"new_prefill":8884,"score_linear":70776.0,"score_lmetric":42546},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":27221,"ongoing_decode_tokens":0,"pending_prefill_tokens":27221,"num_requests":1,"active_p_offloads":0,"cached_blocks":344,"cache_hit":512,"new_prefill":8372,"score_linear":26709.0,"score_lmetric":35593},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":38253,"ongoing_decode_tokens":33524,"pending_prefill_tokens":4729,"num_requests":2,"active_p_offloads":0,"cached_blocks":506,"cache_hit":0,"new_prefill":8884,"score_linear":38253.0,"score_lmetric":27226},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":388,"cache_hit":0,"new_prefill":8884,"score_linear":191531.0,"score_lmetric":568188}],"chosen_score_linear":25187.0,"chosen_score_lmetric":8884,"t_first_token":348448.241816464,"t_first_token_unix":1779779313.2557597,"t_done":348451.648835218,"t_done_unix":1779779316.6627784},{"request_id":"1268630:1:1268630:98","session_id":"1268630","input_length":102074,"t_proxy_recv":348416.313658448,"t_decision_unix":1779779281.3276002,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":false,"cache_hit":3584,"estimated_new_tokens":98490,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":102074,"score_linear":135235.0,"score_lmetric":430586},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":265,"cache_hit":3584,"new_prefill":98490,"score_linear":-3584.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":102074,"score_linear":126924.0,"score_lmetric":210054},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":4608,"new_prefill":97466,"score_linear":10675.0,"score_lmetric":97466},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":29937,"ongoing_decode_tokens":27639,"pending_prefill_tokens":2298,"num_requests":1,"active_p_offloads":0,"cached_blocks":279,"cache_hit":0,"new_prefill":102074,"score_linear":29937.0,"score_lmetric":104372},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":102074,"score_linear":125063.0,"score_lmetric":438914},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":18638,"ongoing_decode_tokens":0,"pending_prefill_tokens":18638,"num_requests":1,"active_p_offloads":0,"cached_blocks":306,"cache_hit":0,"new_prefill":102074,"score_linear":18638.0,"score_lmetric":120712},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":17466,"ongoing_decode_tokens":17466,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":206,"cache_hit":0,"new_prefill":102074,"score_linear":17466.0,"score_lmetric":102074}],"chosen_score_linear":-3584.0,"chosen_score_lmetric":0,"t_first_token":348450.390739327,"t_first_token_unix":1779779315.404683,"t_done":348452.667269813,"t_done_unix":1779779317.6812139},{"request_id":"1270606:1:1270606:113","session_id":"1270606","input_length":93430,"t_proxy_recv":348422.998077735,"t_decision_unix":1779779288.0120187,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.375,"fallback_score":93430,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":93430,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":93430,"score_linear":135235.0,"score_lmetric":413298},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":93430,"score_linear":102074.0,"score_lmetric":191920},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":93430,"score_linear":126924.0,"score_lmetric":201410},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":93430,"score_linear":15283.0,"score_lmetric":93430},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":55566,"ongoing_decode_tokens":27639,"pending_prefill_tokens":27927,"num_requests":1,"active_p_offloads":0,"cached_blocks":309,"cache_hit":0,"new_prefill":93430,"score_linear":55566.0,"score_lmetric":121357},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":93430,"score_linear":125063.0,"score_lmetric":421626},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":86709,"ongoing_decode_tokens":28906,"pending_prefill_tokens":7115,"num_requests":2,"active_p_offloads":0,"cached_blocks":342,"cache_hit":0,"new_prefill":93430,"score_linear":86709.0,"score_lmetric":201090},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":11128,"ongoing_decode_tokens":0,"pending_prefill_tokens":11128,"num_requests":1,"active_p_offloads":0,"cached_blocks":288,"cache_hit":0,"new_prefill":93430,"score_linear":11128.0,"score_lmetric":104558}],"chosen_score_linear":15283.0,"chosen_score_lmetric":93430,"t_first_token":348451.832164238,"t_first_token_unix":1779779316.8461075,"t_done":348453.779375925,"t_done_unix":1779779318.7933192},{"request_id":"1277428:1:1277428:149","session_id":"1277428","input_length":27221,"t_proxy_recv":348447.094633112,"t_decision_unix":1779779312.1085744,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.625,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":27221,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":371,"cache_hit":0,"new_prefill":27221,"score_linear":25187.0,"score_lmetric":27221},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":265,"cache_hit":3584,"new_prefill":23637,"score_linear":98490.0,"score_lmetric":122127},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":27221,"score_linear":150574.0,"score_lmetric":290054},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":108713,"ongoing_decode_tokens":15283,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":17408,"new_prefill":9813,"score_linear":91305.0,"score_lmetric":206486},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":70776,"ongoing_decode_tokens":38931,"pending_prefill_tokens":12389,"num_requests":2,"active_p_offloads":0,"cached_blocks":480,"cache_hit":0,"new_prefill":27221,"score_linear":70776.0,"score_lmetric":79220},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":344,"cache_hit":0,"new_prefill":27221,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":33524,"ongoing_decode_tokens":33524,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":506,"cache_hit":0,"new_prefill":27221,"score_linear":33524.0,"score_lmetric":27221},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":388,"cache_hit":0,"new_prefill":27221,"score_linear":191531.0,"score_lmetric":641536}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348450.557177857,"t_first_token_unix":1779779315.5711217,"t_done":348454.687962512,"t_done_unix":1779779319.7019062},{"request_id":"1268831:4:1279168:158","session_id":"1268831","input_length":32346,"t_proxy_recv":348453.162444557,"t_decision_unix":1779779318.1763852,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":31744,"affinity_cache_ratio":0.9813887343102702,"affinity_num_requests":0,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"cache_hit":31744,"estimated_new_tokens":602,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":389,"cache_hit":0,"new_prefill":32346,"score_linear":25187.0,"score_lmetric":32346},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":58752,"ongoing_decode_tokens":58752,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":457,"cache_hit":0,"new_prefill":32346,"score_linear":58752.0,"score_lmetric":32346},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":32346,"score_linear":150574.0,"score_lmetric":300304},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":108713,"ongoing_decode_tokens":108713,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":32346,"score_linear":108713.0,"score_lmetric":64692},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":522,"cache_hit":31744,"new_prefill":602,"score_linear":-4105.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":145318,"ongoing_decode_tokens":27221,"pending_prefill_tokens":4433,"num_requests":2,"active_p_offloads":0,"cached_blocks":344,"cache_hit":0,"new_prefill":32346,"score_linear":145318.0,"score_lmetric":73558},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":72302,"ongoing_decode_tokens":24222,"pending_prefill_tokens":39376,"num_requests":3,"active_p_offloads":0,"cached_blocks":516,"cache_hit":0,"new_prefill":32346,"score_linear":72302.0,"score_lmetric":215166},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":388,"cache_hit":0,"new_prefill":32346,"score_linear":191531.0,"score_lmetric":662036}],"chosen_score_linear":-4105.0,"chosen_score_lmetric":0,"t_first_token":348453.289021641,"t_first_token_unix":1779779318.302965,"t_done":348454.901457243,"t_done_unix":1779779319.9154007},{"request_id":"1253804:5:1278488:156","session_id":"1253804","input_length":58752,"t_proxy_recv":348450.82964654,"t_decision_unix":1779779315.843588,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":57344,"affinity_cache_ratio":0.9760348583877996,"affinity_num_requests":1,"avg_num_requests":2.125,"fallback_score":null,"tie_break_used":false,"cache_hit":57344,"estimated_new_tokens":1408,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":34071,"ongoing_decode_tokens":34071,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":371,"cache_hit":0,"new_prefill":58752,"score_linear":34071.0,"score_lmetric":117504},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":102074,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":265,"cache_hit":57344,"new_prefill":1408,"score_linear":44730.0,"score_lmetric":1408},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":58752,"score_linear":150574.0,"score_lmetric":353116},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":108713,"ongoing_decode_tokens":15283,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":3584,"new_prefill":55168,"score_linear":105129.0,"score_lmetric":297196},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":82057,"ongoing_decode_tokens":82057,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":497,"cache_hit":0,"new_prefill":58752,"score_linear":82057.0,"score_lmetric":117504},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":145318,"ongoing_decode_tokens":27221,"pending_prefill_tokens":4433,"num_requests":2,"active_p_offloads":0,"cached_blocks":344,"cache_hit":0,"new_prefill":58752,"score_linear":145318.0,"score_lmetric":126370},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":55652,"ongoing_decode_tokens":0,"pending_prefill_tokens":55652,"num_requests":2,"active_p_offloads":0,"cached_blocks":516,"cache_hit":0,"new_prefill":58752,"score_linear":55652.0,"score_lmetric":228808},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":388,"cache_hit":0,"new_prefill":58752,"score_linear":191531.0,"score_lmetric":767660}],"chosen_score_linear":44730.0,"chosen_score_lmetric":1408,"t_first_token":348451.370112843,"t_first_token_unix":1779779316.3840563,"t_done":348456.043143303,"t_done_unix":1779779321.0570867},{"request_id":"1276498:2:1278996:157","session_id":"1276498","input_length":16650,"t_proxy_recv":348452.676339038,"t_decision_unix":1779779317.6902804,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":8704,"affinity_cache_ratio":0.5227627627627628,"affinity_num_requests":2,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"cache_hit":8704,"estimated_new_tokens":7946,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":389,"cache_hit":0,"new_prefill":16650,"score_linear":25187.0,"score_lmetric":16650},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":58752,"ongoing_decode_tokens":58752,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":457,"cache_hit":0,"new_prefill":16650,"score_linear":58752.0,"score_lmetric":16650},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":16650,"score_linear":150574.0,"score_lmetric":268912},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":108713,"ongoing_decode_tokens":108713,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":16650,"score_linear":108713.0,"score_lmetric":33300},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":522,"cache_hit":0,"new_prefill":16650,"score_linear":27639.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":145318,"ongoing_decode_tokens":27221,"pending_prefill_tokens":4433,"num_requests":2,"active_p_offloads":0,"cached_blocks":344,"cache_hit":0,"new_prefill":16650,"score_linear":145318.0,"score_lmetric":42166},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":55652,"ongoing_decode_tokens":24222,"pending_prefill_tokens":31430,"num_requests":2,"active_p_offloads":0,"cached_blocks":516,"cache_hit":8704,"new_prefill":7946,"score_linear":46948.0,"score_lmetric":78752},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":388,"cache_hit":0,"new_prefill":16650,"score_linear":191531.0,"score_lmetric":599252}],"chosen_score_linear":46948.0,"chosen_score_lmetric":78752,"t_first_token":348456.848094942,"t_first_token_unix":1779779321.862038,"t_done":348457.193152139,"t_done_unix":1779779322.2070956},{"request_id":"1260327:4:1277974:154","session_id":"1260327","input_length":31430,"t_proxy_recv":348449.146963829,"t_decision_unix":1779779314.1609046,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":7,"chosen_idx":6,"affinity_cache_hit":31232,"affinity_cache_ratio":0.9937002863506205,"affinity_num_requests":4,"avg_num_requests":1.875,"fallback_score":55652,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":31430,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":34071,"ongoing_decode_tokens":34071,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":371,"cache_hit":0,"new_prefill":31430,"score_linear":34071.0,"score_lmetric":62860},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":31430,"score_linear":102074.0,"score_lmetric":129920},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":31430,"score_linear":150574.0,"score_lmetric":298472},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":108713,"ongoing_decode_tokens":15283,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":31430,"score_linear":108713.0,"score_lmetric":249720},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":82057,"ongoing_decode_tokens":82057,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":497,"cache_hit":0,"new_prefill":31430,"score_linear":82057.0,"score_lmetric":62860},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":27221,"ongoing_decode_tokens":0,"pending_prefill_tokens":27221,"num_requests":1,"active_p_offloads":0,"cached_blocks":344,"cache_hit":0,"new_prefill":31430,"score_linear":27221.0,"score_lmetric":58651},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":24222,"ongoing_decode_tokens":0,"pending_prefill_tokens":24222,"num_requests":1,"active_p_offloads":0,"cached_blocks":516,"cache_hit":0,"new_prefill":31430,"score_linear":24222.0,"score_lmetric":55652},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":388,"cache_hit":31232,"new_prefill":198,"score_linear":160299.0,"score_lmetric":533444}],"chosen_score_linear":24222.0,"chosen_score_lmetric":55652,"t_first_token":348456.147743805,"t_first_token_unix":1779779321.1616874,"t_done":348457.481641654,"t_done_unix":1779779322.4955847},{"request_id":"1262354:8:1280463:163","session_id":"1262354","input_length":33928,"t_proxy_recv":348457.458038998,"t_decision_unix":1779779322.4719803,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":33792,"affinity_cache_ratio":0.9959915114359821,"affinity_num_requests":2,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"cache_hit":33792,"estimated_new_tokens":136,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":45323,"ongoing_decode_tokens":45323,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":389,"cache_hit":0,"new_prefill":33928,"score_linear":45323.0,"score_lmetric":67856},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":460,"cache_hit":0,"new_prefill":33928,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":33928,"score_linear":150574.0,"score_lmetric":303468},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":540,"cache_hit":0,"new_prefill":33928,"score_linear":15283.0,"score_lmetric":33928},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":140475,"ongoing_decode_tokens":27639,"pending_prefill_tokens":95940,"num_requests":2,"active_p_offloads":0,"cached_blocks":523,"cache_hit":0,"new_prefill":33928,"score_linear":140475.0,"score_lmetric":259736},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":118097,"ongoing_decode_tokens":118097,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":397,"cache_hit":0,"new_prefill":33928,"score_linear":118097.0,"score_lmetric":33928},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":55652,"ongoing_decode_tokens":55652,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":531,"cache_hit":33792,"new_prefill":136,"score_linear":21860.0,"score_lmetric":272},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":388,"cache_hit":28672,"new_prefill":5256,"score_linear":162859.0,"score_lmetric":553676}],"chosen_score_linear":21860.0,"chosen_score_lmetric":272,"t_first_token":348457.528195946,"t_first_token_unix":1779779322.542139,"t_done":348458.020078557,"t_done_unix":1779779323.0340219},{"request_id":"1279412:1:1279412:160","session_id":"1279412","input_length":20136,"t_proxy_recv":348454.144292866,"t_decision_unix":1779779319.1582341,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.875,"fallback_score":20136,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":20136,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":389,"cache_hit":0,"new_prefill":20136,"score_linear":25187.0,"score_lmetric":20136},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":58752,"ongoing_decode_tokens":58752,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":457,"cache_hit":0,"new_prefill":20136,"score_linear":58752.0,"score_lmetric":20136},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":20136,"score_linear":150574.0,"score_lmetric":275884},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":540,"cache_hit":0,"new_prefill":20136,"score_linear":15283.0,"score_lmetric":20136},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":59985,"ongoing_decode_tokens":59985,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":522,"cache_hit":0,"new_prefill":20136,"score_linear":59985.0,"score_lmetric":20136},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":145318,"ongoing_decode_tokens":145318,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":344,"cache_hit":0,"new_prefill":20136,"score_linear":145318.0,"score_lmetric":40272},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":72302,"ongoing_decode_tokens":24222,"pending_prefill_tokens":39376,"num_requests":3,"active_p_offloads":0,"cached_blocks":516,"cache_hit":12800,"new_prefill":7336,"score_linear":59502.0,"score_lmetric":140136},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":388,"cache_hit":12800,"new_prefill":7336,"score_linear":178731.0,"score_lmetric":561996}],"chosen_score_linear":25187.0,"chosen_score_lmetric":20136,"t_first_token":348456.25715613,"t_first_token_unix":1779779321.2710993,"t_done":348458.235463989,"t_done_unix":1779779323.2494075},{"request_id":"1258859:3:1278118:155","session_id":"1258859","input_length":118097,"t_proxy_recv":348449.667621124,"t_decision_unix":1779779314.6815624,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":113664,"affinity_cache_ratio":0.962463060026927,"affinity_num_requests":1,"avg_num_requests":2.0,"fallback_score":null,"tie_break_used":false,"cache_hit":113664,"estimated_new_tokens":4433,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":34071,"ongoing_decode_tokens":34071,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":371,"cache_hit":0,"new_prefill":118097,"score_linear":34071.0,"score_lmetric":236194},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":118097,"score_linear":102074.0,"score_lmetric":216587},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":118097,"score_linear":150574.0,"score_lmetric":471806},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":108713,"ongoing_decode_tokens":15283,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":118097,"score_linear":108713.0,"score_lmetric":423054},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":82057,"ongoing_decode_tokens":82057,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":497,"cache_hit":0,"new_prefill":118097,"score_linear":82057.0,"score_lmetric":236194},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":27221,"ongoing_decode_tokens":0,"pending_prefill_tokens":27221,"num_requests":1,"active_p_offloads":0,"cached_blocks":344,"cache_hit":113664,"new_prefill":4433,"score_linear":-86443.0,"score_lmetric":31654},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":55652,"ongoing_decode_tokens":0,"pending_prefill_tokens":55652,"num_requests":2,"active_p_offloads":0,"cached_blocks":516,"cache_hit":0,"new_prefill":118097,"score_linear":55652.0,"score_lmetric":347498},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":388,"cache_hit":0,"new_prefill":118097,"score_linear":191531.0,"score_lmetric":1005040}],"chosen_score_linear":-86443.0,"chosen_score_lmetric":31654,"t_first_token":348453.48822807,"t_first_token_unix":1779779318.5021718,"t_done":348460.624273127,"t_done_unix":1779779325.638223},{"request_id":"1262727:1:1262727:72","session_id":"1262727","input_length":15283,"t_proxy_recv":348395.553876421,"t_decision_unix":1779779260.567818,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":15283,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":143,"cache_hit":0,"new_prefill":15283,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55989,"ongoing_decode_tokens":0,"pending_prefill_tokens":1205,"num_requests":1,"active_p_offloads":0,"cached_blocks":261,"cache_hit":0,"new_prefill":15283,"score_linear":55989.0,"score_lmetric":16488},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":68131,"ongoing_decode_tokens":0,"pending_prefill_tokens":68131,"num_requests":1,"active_p_offloads":0,"cached_blocks":163,"cache_hit":0,"new_prefill":15283,"score_linear":68131.0,"score_lmetric":83414},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":15283,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":113053,"ongoing_decode_tokens":0,"pending_prefill_tokens":92573,"num_requests":2,"active_p_offloads":0,"cached_blocks":78,"cache_hit":0,"new_prefill":15283,"score_linear":113053.0,"score_lmetric":215712},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":15283,"score_linear":125063.0,"score_lmetric":265332},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":197,"cache_hit":0,"new_prefill":15283,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":21510,"ongoing_decode_tokens":0,"pending_prefill_tokens":21510,"num_requests":1,"active_p_offloads":0,"cached_blocks":86,"cache_hit":0,"new_prefill":15283,"score_linear":21510.0,"score_lmetric":36793}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348396.964229604,"t_first_token_unix":1779779261.9781735,"t_done":348460.778924424,"t_done_unix":1779779325.792868},{"request_id":"1268861:2:1280530:164","session_id":"1268861","input_length":12765,"t_proxy_recv":348457.674413261,"t_decision_unix":1779779322.688354,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":7,"chosen_idx":1,"affinity_cache_hit":12288,"affinity_cache_ratio":0.9626321974148061,"affinity_num_requests":4,"avg_num_requests":1.75,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":12765,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":45323,"ongoing_decode_tokens":45323,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":389,"cache_hit":0,"new_prefill":12765,"score_linear":45323.0,"score_lmetric":25530},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":460,"cache_hit":0,"new_prefill":12765,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":12765,"score_linear":150574.0,"score_lmetric":261142},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":540,"cache_hit":0,"new_prefill":12765,"score_linear":15283.0,"score_lmetric":12765},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":140475,"ongoing_decode_tokens":27639,"pending_prefill_tokens":95940,"num_requests":2,"active_p_offloads":0,"cached_blocks":523,"cache_hit":0,"new_prefill":12765,"score_linear":140475.0,"score_lmetric":217410},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":118097,"ongoing_decode_tokens":118097,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":397,"cache_hit":0,"new_prefill":12765,"score_linear":118097.0,"score_lmetric":12765},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":58150,"ongoing_decode_tokens":58150,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":592,"cache_hit":0,"new_prefill":12765,"score_linear":58150.0,"score_lmetric":25530},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":388,"cache_hit":12288,"new_prefill":477,"score_linear":179243.0,"score_lmetric":534560}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348458.737427299,"t_first_token_unix":1779779323.751371,"t_done":348461.058205917,"t_done_unix":1779779326.0721498},{"request_id":"1277909:1:1277909:153","session_id":"1277909","input_length":24222,"t_proxy_recv":348448.948839841,"t_decision_unix":1779779313.962781,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.875,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":24222,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":34071,"ongoing_decode_tokens":34071,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":371,"cache_hit":4096,"new_prefill":20126,"score_linear":29975.0,"score_lmetric":40252},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":24222,"score_linear":102074.0,"score_lmetric":122712},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":24222,"score_linear":150574.0,"score_lmetric":284056},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":108713,"ongoing_decode_tokens":15283,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":24222,"score_linear":108713.0,"score_lmetric":235304},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":93349,"ongoing_decode_tokens":93349,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":480,"cache_hit":0,"new_prefill":24222,"score_linear":93349.0,"score_lmetric":72666},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":27221,"ongoing_decode_tokens":0,"pending_prefill_tokens":27221,"num_requests":1,"active_p_offloads":0,"cached_blocks":344,"cache_hit":0,"new_prefill":24222,"score_linear":27221.0,"score_lmetric":51443},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":516,"cache_hit":0,"new_prefill":24222,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":388,"cache_hit":7680,"new_prefill":16542,"score_linear":183851.0,"score_lmetric":598820}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348451.747737906,"t_first_token_unix":1779779316.7616813,"t_done":348461.366187218,"t_done_unix":1779779326.380131},{"request_id":"1281887:1:1281887:171","session_id":"1281887","input_length":9370,"t_proxy_recv":348462.315766916,"t_decision_unix":1779779327.3297083,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":9370,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":51613,"ongoing_decode_tokens":51613,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":428,"cache_hit":0,"new_prefill":9370,"score_linear":51613.0,"score_lmetric":18740},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":485,"cache_hit":0,"new_prefill":9370,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":9370,"score_linear":150574.0,"score_lmetric":254352},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":575,"cache_hit":0,"new_prefill":9370,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":140475,"ongoing_decode_tokens":27639,"pending_prefill_tokens":95940,"num_requests":2,"active_p_offloads":0,"cached_blocks":523,"cache_hit":0,"new_prefill":9370,"score_linear":140475.0,"score_lmetric":210620},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":406,"cache_hit":0,"new_prefill":9370,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":34028,"ongoing_decode_tokens":34028,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":640,"cache_hit":0,"new_prefill":9370,"score_linear":34028.0,"score_lmetric":9370},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":388,"cache_hit":0,"new_prefill":9370,"score_linear":191531.0,"score_lmetric":570132}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348463.026208321,"t_first_token_unix":1779779328.0401514,"t_done":348463.193411157,"t_done_unix":1779779328.2073545},{"request_id":"1282082:1:1282082:173","session_id":"1282082","input_length":4972,"t_proxy_recv":348463.018847829,"t_decision_unix":1779779328.0327892,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.625,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":4972,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":51613,"ongoing_decode_tokens":51613,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":428,"cache_hit":0,"new_prefill":4972,"score_linear":51613.0,"score_lmetric":9944},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":9370,"ongoing_decode_tokens":0,"pending_prefill_tokens":9370,"num_requests":1,"active_p_offloads":0,"cached_blocks":485,"cache_hit":0,"new_prefill":4972,"score_linear":9370.0,"score_lmetric":14342},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":4972,"score_linear":150574.0,"score_lmetric":245556},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":575,"cache_hit":0,"new_prefill":4972,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":140475,"ongoing_decode_tokens":27639,"pending_prefill_tokens":95940,"num_requests":2,"active_p_offloads":0,"cached_blocks":523,"cache_hit":0,"new_prefill":4972,"score_linear":140475.0,"score_lmetric":201824},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":406,"cache_hit":0,"new_prefill":4972,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":76051,"ongoing_decode_tokens":34028,"pending_prefill_tokens":10791,"num_requests":2,"active_p_offloads":0,"cached_blocks":640,"cache_hit":0,"new_prefill":4972,"score_linear":76051.0,"score_lmetric":31526},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":388,"cache_hit":0,"new_prefill":4972,"score_linear":191531.0,"score_lmetric":552540}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348463.334250034,"t_first_token_unix":1779779328.3481936,"t_done":348463.504239254,"t_done_unix":1779779328.518183},{"request_id":"1267915:1:1267915:96","session_id":"1267915","input_length":126924,"t_proxy_recv":348413.678639679,"t_decision_unix":1779779278.6925814,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":18944,"estimated_new_tokens":107980,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":126924,"score_linear":135235.0,"score_lmetric":480286},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":126924,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":358,"cache_hit":18944,"new_prefill":107980,"score_linear":-18944.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":126924,"score_linear":15283.0,"score_lmetric":126924},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":267,"cache_hit":0,"new_prefill":126924,"score_linear":27639.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":126924,"score_linear":125063.0,"score_lmetric":488614},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":306,"cache_hit":0,"new_prefill":126924,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":46137,"ongoing_decode_tokens":17466,"pending_prefill_tokens":6143,"num_requests":2,"active_p_offloads":0,"cached_blocks":194,"cache_hit":0,"new_prefill":126924,"score_linear":46137.0,"score_lmetric":266134}],"chosen_score_linear":-18944.0,"chosen_score_lmetric":0,"t_first_token":348463.908408809,"t_first_token_unix":1779779328.9223526,"t_done":348465.224718657,"t_done_unix":1779779330.2386627},{"request_id":"1265128:2:1271655:122","session_id":"1265128","input_length":23650,"t_proxy_recv":348426.510141609,"t_decision_unix":1779779291.5240831,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":13824,"affinity_cache_ratio":0.5845243128964059,"affinity_num_requests":1,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"cache_hit":13824,"estimated_new_tokens":9826,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":23650,"score_linear":135235.0,"score_lmetric":273738},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":23650,"score_linear":102074.0,"score_lmetric":122140},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126924,"ongoing_decode_tokens":0,"pending_prefill_tokens":107980,"num_requests":1,"active_p_offloads":0,"cached_blocks":358,"cache_hit":13824,"new_prefill":9826,"score_linear":113100.0,"score_lmetric":117806},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":108713,"ongoing_decode_tokens":15283,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":23650,"score_linear":108713.0,"score_lmetric":234160},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":75014,"ongoing_decode_tokens":75014,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":309,"cache_hit":0,"new_prefill":23650,"score_linear":75014.0,"score_lmetric":47300},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":113923,"ongoing_decode_tokens":113923,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":23650,"score_linear":113923.0,"score_lmetric":47300},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":46612,"ongoing_decode_tokens":28906,"pending_prefill_tokens":17706,"num_requests":2,"active_p_offloads":0,"cached_blocks":355,"cache_hit":0,"new_prefill":23650,"score_linear":46612.0,"score_lmetric":82712},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":64508,"ongoing_decode_tokens":28385,"pending_prefill_tokens":21787,"num_requests":3,"active_p_offloads":0,"cached_blocks":309,"cache_hit":0,"new_prefill":23650,"score_linear":64508.0,"score_lmetric":136311}],"chosen_score_linear":113100.0,"chosen_score_lmetric":117806,"t_first_token":348464.405688893,"t_first_token_unix":1779779329.4196327,"t_done":348465.584268704,"t_done_unix":1779779330.598212},{"request_id":"1262354:9:1281632:169","session_id":"1262354","input_length":34028,"t_proxy_recv":348461.450719455,"t_decision_unix":1779779326.4646606,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":33792,"affinity_cache_ratio":0.9930645350887505,"affinity_num_requests":0,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":33792,"estimated_new_tokens":236,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":51613,"ongoing_decode_tokens":51613,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":428,"cache_hit":12800,"new_prefill":21228,"score_linear":38813.0,"score_lmetric":42456},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":485,"cache_hit":0,"new_prefill":34028,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":34028,"score_linear":150574.0,"score_lmetric":303668},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":575,"cache_hit":0,"new_prefill":34028,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":140475,"ongoing_decode_tokens":27639,"pending_prefill_tokens":95940,"num_requests":2,"active_p_offloads":0,"cached_blocks":523,"cache_hit":0,"new_prefill":34028,"score_linear":140475.0,"score_lmetric":259936},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":406,"cache_hit":0,"new_prefill":34028,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":640,"cache_hit":33792,"new_prefill":236,"score_linear":-33792.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":388,"cache_hit":28672,"new_prefill":5356,"score_linear":162859.0,"score_lmetric":554076}],"chosen_score_linear":-33792.0,"chosen_score_lmetric":0,"t_first_token":348461.516157457,"t_first_token_unix":1779779326.5301008,"t_done":348465.960907088,"t_done_unix":1779779330.9748511},{"request_id":"1260327:5:1281915:172","session_id":"1260327","input_length":42023,"t_proxy_recv":348462.431574814,"t_decision_unix":1779779327.4455163,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":31232,"affinity_cache_ratio":0.7432120505437498,"affinity_num_requests":1,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":31232,"estimated_new_tokens":10791,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":51613,"ongoing_decode_tokens":51613,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":428,"cache_hit":0,"new_prefill":42023,"score_linear":51613.0,"score_lmetric":84046},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":9370,"ongoing_decode_tokens":0,"pending_prefill_tokens":9370,"num_requests":1,"active_p_offloads":0,"cached_blocks":485,"cache_hit":0,"new_prefill":42023,"score_linear":9370.0,"score_lmetric":51393},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":42023,"score_linear":150574.0,"score_lmetric":319658},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":575,"cache_hit":0,"new_prefill":42023,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":140475,"ongoing_decode_tokens":27639,"pending_prefill_tokens":95940,"num_requests":2,"active_p_offloads":0,"cached_blocks":523,"cache_hit":0,"new_prefill":42023,"score_linear":140475.0,"score_lmetric":275926},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":406,"cache_hit":0,"new_prefill":42023,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":34028,"ongoing_decode_tokens":34028,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":640,"cache_hit":31232,"new_prefill":10791,"score_linear":2796.0,"score_lmetric":10791},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":388,"cache_hit":31232,"new_prefill":10791,"score_linear":160299.0,"score_lmetric":575816}],"chosen_score_linear":2796.0,"chosen_score_lmetric":10791,"t_first_token":348465.188259302,"t_first_token_unix":1779779330.2022028,"t_done":348466.047356392,"t_done_unix":1779779331.0612996},{"request_id":"1279412:2:1281113:167","session_id":"1279412","input_length":26426,"t_proxy_recv":348459.93494571,"t_decision_unix":1779779324.9488866,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":19968,"affinity_cache_ratio":0.7556194656777416,"affinity_num_requests":1,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"cache_hit":19968,"estimated_new_tokens":6458,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":428,"cache_hit":19968,"new_prefill":6458,"score_linear":5219.0,"score_lmetric":6458},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":12765,"ongoing_decode_tokens":12765,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":460,"cache_hit":0,"new_prefill":26426,"score_linear":12765.0,"score_lmetric":26426},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":26426,"score_linear":150574.0,"score_lmetric":288464},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":540,"cache_hit":0,"new_prefill":26426,"score_linear":15283.0,"score_lmetric":26426},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":140475,"ongoing_decode_tokens":27639,"pending_prefill_tokens":95940,"num_requests":2,"active_p_offloads":0,"cached_blocks":523,"cache_hit":0,"new_prefill":26426,"score_linear":140475.0,"score_lmetric":244732},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":118097,"ongoing_decode_tokens":118097,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":397,"cache_hit":0,"new_prefill":26426,"score_linear":118097.0,"score_lmetric":26426},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":24222,"ongoing_decode_tokens":24222,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":592,"cache_hit":12800,"new_prefill":13626,"score_linear":11422.0,"score_lmetric":13626},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":388,"cache_hit":12800,"new_prefill":13626,"score_linear":178731.0,"score_lmetric":587156}],"chosen_score_linear":5219.0,"chosen_score_lmetric":6458,"t_first_token":348461.066319473,"t_first_token_unix":1779779326.080263,"t_done":348466.342768977,"t_done_unix":1779779331.3567123},{"request_id":"1277533:2:1282872:175","session_id":"1277533","input_length":16299,"t_proxy_recv":348465.733011847,"t_decision_unix":1779779330.7469532,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":6,"chosen_idx":1,"affinity_cache_hit":4608,"affinity_cache_ratio":0.2827167310877968,"affinity_num_requests":2,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":16299,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":51613,"ongoing_decode_tokens":51613,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":428,"cache_hit":0,"new_prefill":16299,"score_linear":51613.0,"score_lmetric":32598},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":503,"cache_hit":0,"new_prefill":16299,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":566,"cache_hit":0,"new_prefill":16299,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":575,"cache_hit":0,"new_prefill":16299,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":140475,"ongoing_decode_tokens":27639,"pending_prefill_tokens":95940,"num_requests":2,"active_p_offloads":0,"cached_blocks":523,"cache_hit":0,"new_prefill":16299,"score_linear":140475.0,"score_lmetric":224478},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":415,"cache_hit":0,"new_prefill":16299,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":76051,"ongoing_decode_tokens":76051,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":640,"cache_hit":4608,"new_prefill":11691,"score_linear":71443.0,"score_lmetric":23382},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":388,"cache_hit":0,"new_prefill":16299,"score_linear":191531.0,"score_lmetric":597848}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348467.223676705,"t_first_token_unix":1779779332.2376206,"t_done":348467.241686661,"t_done_unix":1779779332.25563},{"request_id":"1283642:1:1283642:178","session_id":"1283642","input_length":8510,"t_proxy_recv":348468.169379176,"t_decision_unix":1779779333.1833205,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":8510,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":441,"cache_hit":0,"new_prefill":8510,"score_linear":25187.0,"score_lmetric":8510},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":534,"cache_hit":0,"new_prefill":8510,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":566,"cache_hit":0,"new_prefill":8510,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":575,"cache_hit":0,"new_prefill":8510,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":140475,"ongoing_decode_tokens":27639,"pending_prefill_tokens":95940,"num_requests":2,"active_p_offloads":0,"cached_blocks":523,"cache_hit":0,"new_prefill":8510,"score_linear":140475.0,"score_lmetric":208900},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":118736,"ongoing_decode_tokens":118736,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":415,"cache_hit":0,"new_prefill":8510,"score_linear":118736.0,"score_lmetric":8510},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":661,"cache_hit":0,"new_prefill":8510,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":388,"cache_hit":0,"new_prefill":8510,"score_linear":191531.0,"score_lmetric":566692}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348468.800297546,"t_first_token_unix":1779779333.814241,"t_done":348469.113389234,"t_done_unix":1779779334.1273324},{"request_id":"1258859:4:1282900:176","session_id":"1258859","input_length":118736,"t_proxy_recv":348465.807537284,"t_decision_unix":1779779330.8214786,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":118272,"affinity_cache_ratio":0.9960921708664601,"affinity_num_requests":0,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":118272,"estimated_new_tokens":464,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":51613,"ongoing_decode_tokens":51613,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":428,"cache_hit":0,"new_prefill":118736,"score_linear":51613.0,"score_lmetric":237472},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":16299,"ongoing_decode_tokens":0,"pending_prefill_tokens":16299,"num_requests":1,"active_p_offloads":0,"cached_blocks":503,"cache_hit":0,"new_prefill":118736,"score_linear":16299.0,"score_lmetric":135035},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":566,"cache_hit":0,"new_prefill":118736,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":575,"cache_hit":0,"new_prefill":118736,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":140475,"ongoing_decode_tokens":27639,"pending_prefill_tokens":95940,"num_requests":2,"active_p_offloads":0,"cached_blocks":523,"cache_hit":0,"new_prefill":118736,"score_linear":140475.0,"score_lmetric":429352},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":415,"cache_hit":118272,"new_prefill":464,"score_linear":-118272.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":76051,"ongoing_decode_tokens":76051,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":640,"cache_hit":0,"new_prefill":118736,"score_linear":76051.0,"score_lmetric":237472},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":388,"cache_hit":0,"new_prefill":118736,"score_linear":191531.0,"score_lmetric":1007596}],"chosen_score_linear":-118272.0,"chosen_score_lmetric":0,"t_first_token":348465.979595707,"t_first_token_unix":1779779330.993539,"t_done":348469.931233308,"t_done_unix":1779779334.9451797},{"request_id":"1262354:10:1283773:180","session_id":"1262354","input_length":34282,"t_proxy_recv":348468.603827267,"t_decision_unix":1779779333.6177688,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":33792,"affinity_cache_ratio":0.9857067849016977,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":33792,"estimated_new_tokens":490,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":441,"cache_hit":12800,"new_prefill":21482,"score_linear":12387.0,"score_lmetric":21482},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":534,"cache_hit":0,"new_prefill":34282,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":8510,"ongoing_decode_tokens":0,"pending_prefill_tokens":8510,"num_requests":1,"active_p_offloads":0,"cached_blocks":566,"cache_hit":0,"new_prefill":34282,"score_linear":8510.0,"score_lmetric":42792},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":575,"cache_hit":0,"new_prefill":34282,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":140475,"ongoing_decode_tokens":27639,"pending_prefill_tokens":95940,"num_requests":2,"active_p_offloads":0,"cached_blocks":523,"cache_hit":0,"new_prefill":34282,"score_linear":140475.0,"score_lmetric":260444},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":118736,"ongoing_decode_tokens":118736,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":415,"cache_hit":0,"new_prefill":34282,"score_linear":118736.0,"score_lmetric":34282},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":30965,"ongoing_decode_tokens":0,"pending_prefill_tokens":6389,"num_requests":1,"active_p_offloads":0,"cached_blocks":661,"cache_hit":33792,"new_prefill":490,"score_linear":-2827.0,"score_lmetric":6879},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":388,"cache_hit":28672,"new_prefill":5610,"score_linear":162859.0,"score_lmetric":555092}],"chosen_score_linear":-2827.0,"chosen_score_lmetric":6879,"t_first_token":348469.597944828,"t_first_token_unix":1779779334.6118884,"t_done":348470.168814471,"t_done_unix":1779779335.182758},{"request_id":"1242838:4:1265709:87","session_id":"1242838","input_length":25187,"t_proxy_recv":348406.139043717,"t_decision_unix":1779779271.152985,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":22016,"affinity_cache_ratio":0.8741017191408266,"affinity_num_requests":1,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":22016,"estimated_new_tokens":3171,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":110048,"ongoing_decode_tokens":0,"pending_prefill_tokens":110048,"num_requests":1,"active_p_offloads":0,"cached_blocks":156,"cache_hit":22016,"new_prefill":3171,"score_linear":88032.0,"score_lmetric":113219},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55989,"ongoing_decode_tokens":55989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":261,"cache_hit":0,"new_prefill":25187,"score_linear":55989.0,"score_lmetric":25187},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":323,"cache_hit":0,"new_prefill":25187,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":71575,"ongoing_decode_tokens":15283,"pending_prefill_tokens":56292,"num_requests":2,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":25187,"score_linear":71575.0,"score_lmetric":162958},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":258,"cache_hit":0,"new_prefill":25187,"score_linear":27639.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":25187,"score_linear":125063.0,"score_lmetric":285140},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":50783,"ongoing_decode_tokens":0,"pending_prefill_tokens":50783,"num_requests":1,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":25187,"score_linear":50783.0,"score_lmetric":75970},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":45884,"ongoing_decode_tokens":0,"pending_prefill_tokens":23868,"num_requests":2,"active_p_offloads":0,"cached_blocks":141,"cache_hit":0,"new_prefill":25187,"score_linear":45884.0,"score_lmetric":98110}],"chosen_score_linear":88032.0,"chosen_score_lmetric":113219,"t_first_token":348442.411237005,"t_first_token_unix":1779779307.4251833,"t_done":348473.492160041,"t_done_unix":1779779338.5061064},{"request_id":"1277909:2:1283679:179","session_id":"1277909","input_length":30965,"t_proxy_recv":348468.277337148,"t_decision_unix":1779779333.2912788,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":24576,"affinity_cache_ratio":0.7936702728887454,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":24576,"estimated_new_tokens":6389,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":441,"cache_hit":4096,"new_prefill":26869,"score_linear":21091.0,"score_lmetric":26869},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":534,"cache_hit":0,"new_prefill":30965,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":8510,"ongoing_decode_tokens":0,"pending_prefill_tokens":8510,"num_requests":1,"active_p_offloads":0,"cached_blocks":566,"cache_hit":0,"new_prefill":30965,"score_linear":8510.0,"score_lmetric":39475},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":575,"cache_hit":0,"new_prefill":30965,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":140475,"ongoing_decode_tokens":27639,"pending_prefill_tokens":95940,"num_requests":2,"active_p_offloads":0,"cached_blocks":523,"cache_hit":0,"new_prefill":30965,"score_linear":140475.0,"score_lmetric":253810},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":118736,"ongoing_decode_tokens":118736,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":415,"cache_hit":0,"new_prefill":30965,"score_linear":118736.0,"score_lmetric":30965},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":661,"cache_hit":24576,"new_prefill":6389,"score_linear":-24576.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":388,"cache_hit":7680,"new_prefill":23285,"score_linear":183851.0,"score_lmetric":625792}],"chosen_score_linear":-24576.0,"chosen_score_lmetric":0,"t_first_token":348469.56234849,"t_first_token_unix":1779779334.576292,"t_done":348474.766637318,"t_done_unix":1779779339.7805808},{"request_id":"1279412:3:1285475:187","session_id":"1279412","input_length":27077,"t_proxy_recv":348474.637935267,"t_decision_unix":1779779339.6518764,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":26624,"affinity_cache_ratio":0.9832699338922333,"affinity_num_requests":0,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":26624,"estimated_new_tokens":453,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":451,"cache_hit":26624,"new_prefill":453,"score_linear":-26624.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":534,"cache_hit":0,"new_prefill":27077,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":582,"cache_hit":0,"new_prefill":27077,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":49619,"ongoing_decode_tokens":0,"pending_prefill_tokens":49619,"num_requests":1,"active_p_offloads":0,"cached_blocks":575,"cache_hit":0,"new_prefill":27077,"score_linear":49619.0,"score_lmetric":76696},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":182739,"ongoing_decode_tokens":27639,"pending_prefill_tokens":105948,"num_requests":3,"active_p_offloads":0,"cached_blocks":523,"cache_hit":0,"new_prefill":27077,"score_linear":182739.0,"score_lmetric":399075},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":416,"cache_hit":0,"new_prefill":27077,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":65318,"ongoing_decode_tokens":65318,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":662,"cache_hit":12800,"new_prefill":14277,"score_linear":52518.0,"score_lmetric":28554},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":388,"cache_hit":12800,"new_prefill":14277,"score_linear":178731.0,"score_lmetric":589760}],"chosen_score_linear":-26624.0,"chosen_score_lmetric":0,"t_first_token":348474.704303184,"t_first_token_unix":1779779339.7182467,"t_done":348474.772357935,"t_done_unix":1779779339.7863011},{"request_id":"1285800:1:1285800:191","session_id":"1285800","input_length":4510,"t_proxy_recv":348475.744922084,"t_decision_unix":1779779340.7588632,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":4510,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":451,"cache_hit":0,"new_prefill":4510,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":534,"cache_hit":0,"new_prefill":4510,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":14192,"ongoing_decode_tokens":0,"pending_prefill_tokens":6000,"num_requests":1,"active_p_offloads":0,"cached_blocks":582,"cache_hit":0,"new_prefill":4510,"score_linear":14192.0,"score_lmetric":10510},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":49619,"ongoing_decode_tokens":0,"pending_prefill_tokens":49619,"num_requests":1,"active_p_offloads":0,"cached_blocks":575,"cache_hit":0,"new_prefill":4510,"score_linear":49619.0,"score_lmetric":54129},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":182739,"ongoing_decode_tokens":27639,"pending_prefill_tokens":105948,"num_requests":3,"active_p_offloads":0,"cached_blocks":523,"cache_hit":0,"new_prefill":4510,"score_linear":182739.0,"score_lmetric":331374},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":11,"ongoing_decode_tokens":11,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":416,"cache_hit":0,"new_prefill":4510,"score_linear":11.0,"score_lmetric":4510},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":34353,"ongoing_decode_tokens":34353,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":675,"cache_hit":0,"new_prefill":4510,"score_linear":34353.0,"score_lmetric":4510},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":388,"cache_hit":0,"new_prefill":4510,"score_linear":191531.0,"score_lmetric":550692}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348476.02845161,"t_first_token_unix":1779779341.0423949,"t_done":348476.309873327,"t_done_unix":1779779341.3238173},{"request_id":"1283642:2:1285731:190","session_id":"1283642","input_length":14192,"t_proxy_recv":348475.491310586,"t_decision_unix":1779779340.5052521,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":8192,"affinity_cache_ratio":0.5772266065388951,"affinity_num_requests":0,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":8192,"estimated_new_tokens":6000,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":451,"cache_hit":0,"new_prefill":14192,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":534,"cache_hit":0,"new_prefill":14192,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":582,"cache_hit":8192,"new_prefill":6000,"score_linear":-8192.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":49619,"ongoing_decode_tokens":0,"pending_prefill_tokens":49619,"num_requests":1,"active_p_offloads":0,"cached_blocks":575,"cache_hit":0,"new_prefill":14192,"score_linear":49619.0,"score_lmetric":63811},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":182739,"ongoing_decode_tokens":27639,"pending_prefill_tokens":105948,"num_requests":3,"active_p_offloads":0,"cached_blocks":523,"cache_hit":0,"new_prefill":14192,"score_linear":182739.0,"score_lmetric":360420},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":11,"ongoing_decode_tokens":11,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":416,"cache_hit":0,"new_prefill":14192,"score_linear":11.0,"score_lmetric":14192},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":34353,"ongoing_decode_tokens":34353,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":675,"cache_hit":0,"new_prefill":14192,"score_linear":34353.0,"score_lmetric":14192},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":388,"cache_hit":0,"new_prefill":14192,"score_linear":191531.0,"score_lmetric":589420}],"chosen_score_linear":-8192.0,"chosen_score_lmetric":0,"t_first_token":348476.14179407,"t_first_token_unix":1779779341.1557376,"t_done":348476.532826535,"t_done_unix":1779779341.5467703},{"request_id":"1285558:1:1285558:188","session_id":"1285558","input_length":11,"t_proxy_recv":348474.838911324,"t_decision_unix":1779779339.8528526,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":11,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":451,"cache_hit":0,"new_prefill":11,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":534,"cache_hit":0,"new_prefill":11,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":582,"cache_hit":0,"new_prefill":11,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":49619,"ongoing_decode_tokens":0,"pending_prefill_tokens":49619,"num_requests":1,"active_p_offloads":0,"cached_blocks":575,"cache_hit":0,"new_prefill":11,"score_linear":49619.0,"score_lmetric":49630},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":182739,"ongoing_decode_tokens":27639,"pending_prefill_tokens":105948,"num_requests":3,"active_p_offloads":0,"cached_blocks":523,"cache_hit":0,"new_prefill":11,"score_linear":182739.0,"score_lmetric":317877},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":416,"cache_hit":0,"new_prefill":11,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":34353,"ongoing_decode_tokens":34353,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":675,"cache_hit":0,"new_prefill":11,"score_linear":34353.0,"score_lmetric":11},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":388,"cache_hit":0,"new_prefill":11,"score_linear":191531.0,"score_lmetric":532696}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348474.858178527,"t_first_token_unix":1779779339.872122,"t_done":348476.568592519,"t_done_unix":1779779341.5825355},{"request_id":"1262354:11:1284987:186","session_id":"1262354","input_length":34353,"t_proxy_recv":348473.098915509,"t_decision_unix":1779779338.1128569,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":34304,"affinity_cache_ratio":0.9985736325793962,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":34304,"estimated_new_tokens":49,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":441,"cache_hit":12800,"new_prefill":21553,"score_linear":12387.0,"score_lmetric":21553},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":534,"cache_hit":0,"new_prefill":34353,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":582,"cache_hit":0,"new_prefill":34353,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":49619,"ongoing_decode_tokens":0,"pending_prefill_tokens":49619,"num_requests":1,"active_p_offloads":0,"cached_blocks":575,"cache_hit":0,"new_prefill":34353,"score_linear":49619.0,"score_lmetric":83972},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":182739,"ongoing_decode_tokens":27639,"pending_prefill_tokens":105948,"num_requests":3,"active_p_offloads":0,"cached_blocks":523,"cache_hit":0,"new_prefill":34353,"score_linear":182739.0,"score_lmetric":420903},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":416,"cache_hit":0,"new_prefill":34353,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":30965,"ongoing_decode_tokens":30965,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":662,"cache_hit":34304,"new_prefill":49,"score_linear":-3339.0,"score_lmetric":49},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":388,"cache_hit":28672,"new_prefill":5681,"score_linear":162859.0,"score_lmetric":555376}],"chosen_score_linear":-3339.0,"chosen_score_lmetric":49,"t_first_token":348473.165663877,"t_first_token_unix":1779779338.1796067,"t_done":348478.318247238,"t_done_unix":1779779343.3321905},{"request_id":"1286551:1:1286551:194","session_id":"1286551","input_length":1700,"t_proxy_recv":348478.708526837,"t_decision_unix":1779779343.7224681,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":1700,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":31898,"ongoing_decode_tokens":0,"pending_prefill_tokens":5274,"num_requests":1,"active_p_offloads":0,"cached_blocks":459,"cache_hit":0,"new_prefill":1700,"score_linear":31898.0,"score_lmetric":6974},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":534,"cache_hit":0,"new_prefill":1700,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":593,"cache_hit":0,"new_prefill":1700,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":49619,"ongoing_decode_tokens":0,"pending_prefill_tokens":49619,"num_requests":1,"active_p_offloads":0,"cached_blocks":575,"cache_hit":0,"new_prefill":1700,"score_linear":49619.0,"score_lmetric":51319},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":182739,"ongoing_decode_tokens":27639,"pending_prefill_tokens":105948,"num_requests":3,"active_p_offloads":0,"cached_blocks":523,"cache_hit":0,"new_prefill":1700,"score_linear":182739.0,"score_lmetric":322944},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":416,"cache_hit":0,"new_prefill":1700,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":676,"cache_hit":0,"new_prefill":1700,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":388,"cache_hit":0,"new_prefill":1700,"score_linear":191531.0,"score_lmetric":539452}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348478.853542742,"t_first_token_unix":1779779343.867486,"t_done":348479.048445127,"t_done_unix":1779779344.0623884},{"request_id":"1279412:4:1286447:193","session_id":"1279412","input_length":31898,"t_proxy_recv":348478.312613354,"t_decision_unix":1779779343.3265545,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":26624,"affinity_cache_ratio":0.8346604802808953,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":26624,"estimated_new_tokens":5274,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":459,"cache_hit":26624,"new_prefill":5274,"score_linear":-26624.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":534,"cache_hit":0,"new_prefill":31898,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":593,"cache_hit":0,"new_prefill":31898,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":49619,"ongoing_decode_tokens":0,"pending_prefill_tokens":49619,"num_requests":1,"active_p_offloads":0,"cached_blocks":575,"cache_hit":0,"new_prefill":31898,"score_linear":49619.0,"score_lmetric":81517},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":182739,"ongoing_decode_tokens":27639,"pending_prefill_tokens":105948,"num_requests":3,"active_p_offloads":0,"cached_blocks":523,"cache_hit":0,"new_prefill":31898,"score_linear":182739.0,"score_lmetric":413538},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":416,"cache_hit":0,"new_prefill":31898,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":34353,"ongoing_decode_tokens":34353,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":675,"cache_hit":12800,"new_prefill":19098,"score_linear":21553.0,"score_lmetric":19098},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":388,"cache_hit":12800,"new_prefill":19098,"score_linear":178731.0,"score_lmetric":609044}],"chosen_score_linear":-26624.0,"chosen_score_lmetric":0,"t_first_token":348479.389126135,"t_first_token_unix":1779779344.4030697,"t_done":348481.023658126,"t_done_unix":1779779346.0376017},{"request_id":"1286804:1:1286804:195","session_id":"1286804","input_length":13962,"t_proxy_recv":348479.441557811,"t_decision_unix":1779779344.4554992,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":13962,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":31898,"ongoing_decode_tokens":31898,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":459,"cache_hit":0,"new_prefill":13962,"score_linear":31898.0,"score_lmetric":13962},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":534,"cache_hit":0,"new_prefill":13962,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":596,"cache_hit":0,"new_prefill":13962,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":49619,"ongoing_decode_tokens":49619,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":575,"cache_hit":0,"new_prefill":13962,"score_linear":49619.0,"score_lmetric":13962},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":182739,"ongoing_decode_tokens":27639,"pending_prefill_tokens":105948,"num_requests":3,"active_p_offloads":0,"cached_blocks":523,"cache_hit":0,"new_prefill":13962,"score_linear":182739.0,"score_lmetric":359730},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":416,"cache_hit":0,"new_prefill":13962,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":676,"cache_hit":0,"new_prefill":13962,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":388,"cache_hit":0,"new_prefill":13962,"score_linear":191531.0,"score_lmetric":588500}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348480.63917014,"t_first_token_unix":1779779345.6531134,"t_done":348481.344561206,"t_done_unix":1779779346.3585048},{"request_id":"1287588:1:1287588:198","session_id":"1287588","input_length":9925,"t_proxy_recv":348481.990903374,"t_decision_unix":1779779347.0048444,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":9925,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":469,"cache_hit":0,"new_prefill":9925,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":77805,"ongoing_decode_tokens":0,"pending_prefill_tokens":77805,"num_requests":1,"active_p_offloads":0,"cached_blocks":534,"cache_hit":0,"new_prefill":9925,"score_linear":77805.0,"score_lmetric":87730},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":596,"cache_hit":0,"new_prefill":9925,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":49619,"ongoing_decode_tokens":49619,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":575,"cache_hit":0,"new_prefill":9925,"score_linear":49619.0,"score_lmetric":9925},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":182739,"ongoing_decode_tokens":27639,"pending_prefill_tokens":105948,"num_requests":3,"active_p_offloads":0,"cached_blocks":523,"cache_hit":0,"new_prefill":9925,"score_linear":182739.0,"score_lmetric":347619},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":443,"cache_hit":0,"new_prefill":9925,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":676,"cache_hit":0,"new_prefill":9925,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":191531,"ongoing_decode_tokens":114612,"pending_prefill_tokens":18551,"num_requests":4,"active_p_offloads":0,"cached_blocks":388,"cache_hit":0,"new_prefill":9925,"score_linear":191531.0,"score_lmetric":113904}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348482.75577369,"t_first_token_unix":1779779347.7697175,"t_done":348484.080450755,"t_done_unix":1779779349.0943942},{"request_id":"1288143:1:1288143:200","session_id":"1288143","input_length":448,"t_proxy_recv":348483.95570164,"t_decision_unix":1779779348.969643,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":448,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":9925,"ongoing_decode_tokens":9925,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":469,"cache_hit":0,"new_prefill":448,"score_linear":9925.0,"score_lmetric":448},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":77805,"ongoing_decode_tokens":0,"pending_prefill_tokens":77805,"num_requests":1,"active_p_offloads":0,"cached_blocks":534,"cache_hit":0,"new_prefill":448,"score_linear":77805.0,"score_lmetric":78253},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":596,"cache_hit":0,"new_prefill":448,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":49619,"ongoing_decode_tokens":49619,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":575,"cache_hit":0,"new_prefill":448,"score_linear":49619.0,"score_lmetric":448},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":182739,"ongoing_decode_tokens":115956,"pending_prefill_tokens":17631,"num_requests":3,"active_p_offloads":0,"cached_blocks":523,"cache_hit":0,"new_prefill":448,"score_linear":182739.0,"score_lmetric":54237},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":443,"cache_hit":0,"new_prefill":448,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":676,"cache_hit":0,"new_prefill":448,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":191531,"ongoing_decode_tokens":125251,"pending_prefill_tokens":17640,"num_requests":4,"active_p_offloads":0,"cached_blocks":388,"cache_hit":0,"new_prefill":448,"score_linear":191531.0,"score_lmetric":72352}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348484.001873295,"t_first_token_unix":1779779349.0158165,"t_done":348484.518659873,"t_done_unix":1779779349.5326028},{"request_id":"1284132:1:1284132:182","session_id":"1284132","input_length":49619,"t_proxy_recv":348469.961842343,"t_decision_unix":1779779334.9757836,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":49619,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":441,"cache_hit":0,"new_prefill":49619,"score_linear":25187.0,"score_lmetric":49619},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":534,"cache_hit":0,"new_prefill":49619,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":582,"cache_hit":0,"new_prefill":49619,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":575,"cache_hit":0,"new_prefill":49619,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":140475,"ongoing_decode_tokens":27639,"pending_prefill_tokens":95940,"num_requests":2,"active_p_offloads":0,"cached_blocks":523,"cache_hit":0,"new_prefill":49619,"score_linear":140475.0,"score_lmetric":291118},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":416,"cache_hit":0,"new_prefill":49619,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":65247,"ongoing_decode_tokens":65247,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":661,"cache_hit":0,"new_prefill":49619,"score_linear":65247.0,"score_lmetric":99238},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":388,"cache_hit":0,"new_prefill":49619,"score_linear":191531.0,"score_lmetric":731128}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348479.202742737,"t_first_token_unix":1779779344.2166865,"t_done":348485.921817316,"t_done_unix":1779779350.9357607},{"request_id":"1276136:2:1280145:162","session_id":"1276136","input_length":24519,"t_proxy_recv":348456.453632724,"t_decision_unix":1779779321.4675744,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":16896,"affinity_cache_ratio":0.6890982503364738,"affinity_num_requests":1,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"cache_hit":16896,"estimated_new_tokens":7623,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":45323,"ongoing_decode_tokens":45323,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":389,"cache_hit":0,"new_prefill":24519,"score_linear":45323.0,"score_lmetric":49038},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":460,"cache_hit":0,"new_prefill":24519,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":24519,"score_linear":150574.0,"score_lmetric":284650},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":540,"cache_hit":0,"new_prefill":24519,"score_linear":15283.0,"score_lmetric":24519},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":115956,"ongoing_decode_tokens":27639,"pending_prefill_tokens":88317,"num_requests":1,"active_p_offloads":0,"cached_blocks":523,"cache_hit":16896,"new_prefill":7623,"score_linear":99060.0,"score_lmetric":95940},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":118097,"ongoing_decode_tokens":118097,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":397,"cache_hit":0,"new_prefill":24519,"score_linear":118097.0,"score_lmetric":24519},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":72302,"ongoing_decode_tokens":55652,"pending_prefill_tokens":7946,"num_requests":3,"active_p_offloads":0,"cached_blocks":516,"cache_hit":0,"new_prefill":24519,"score_linear":72302.0,"score_lmetric":97395},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":388,"cache_hit":0,"new_prefill":24519,"score_linear":191531.0,"score_lmetric":630728}],"chosen_score_linear":99060.0,"chosen_score_lmetric":95940,"t_first_token":348484.094306488,"t_first_token_unix":1779779349.1082497,"t_done":348486.667277298,"t_done_unix":1779779351.6812205},{"request_id":"1279994:1:1279994:161","session_id":"1279994","input_length":88317,"t_proxy_recv":348456.059706467,"t_decision_unix":1779779321.073648,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.625,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":88317,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":45323,"ongoing_decode_tokens":25187,"pending_prefill_tokens":20136,"num_requests":2,"active_p_offloads":0,"cached_blocks":389,"cache_hit":0,"new_prefill":88317,"score_linear":45323.0,"score_lmetric":216906},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":460,"cache_hit":0,"new_prefill":88317,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":88317,"score_linear":150574.0,"score_lmetric":412246},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":540,"cache_hit":0,"new_prefill":88317,"score_linear":15283.0,"score_lmetric":88317},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":523,"cache_hit":0,"new_prefill":88317,"score_linear":27639.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":118097,"ongoing_decode_tokens":118097,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":397,"cache_hit":0,"new_prefill":88317,"score_linear":118097.0,"score_lmetric":88317},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":72302,"ongoing_decode_tokens":24222,"pending_prefill_tokens":39376,"num_requests":3,"active_p_offloads":0,"cached_blocks":516,"cache_hit":0,"new_prefill":88317,"score_linear":72302.0,"score_lmetric":383079},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":388,"cache_hit":0,"new_prefill":88317,"score_linear":191531.0,"score_lmetric":885920}],"chosen_score_linear":27639.0,"chosen_score_lmetric":0,"t_first_token":348482.438838351,"t_first_token_unix":1779779347.452782,"t_done":348487.92229594,"t_done_unix":1779779352.9362392},{"request_id":"1276136:3:1280987:166","session_id":"1276136","input_length":25923,"t_proxy_recv":348486.674667718,"t_decision_unix":1779779351.6886091,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":24064,"affinity_cache_ratio":0.9282876210315164,"affinity_num_requests":2,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":24064,"estimated_new_tokens":1859,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":488,"cache_hit":0,"new_prefill":25923,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":77805,"ongoing_decode_tokens":0,"pending_prefill_tokens":77805,"num_requests":1,"active_p_offloads":0,"cached_blocks":534,"cache_hit":0,"new_prefill":25923,"score_linear":77805.0,"score_lmetric":103728},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":596,"cache_hit":0,"new_prefill":25923,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":673,"cache_hit":0,"new_prefill":25923,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":158220,"ongoing_decode_tokens":158220,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":537,"cache_hit":24064,"new_prefill":1859,"score_linear":134156.0,"score_lmetric":3718},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":443,"cache_hit":0,"new_prefill":25923,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":677,"cache_hit":0,"new_prefill":25923,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":191531,"ongoing_decode_tokens":191531,"pending_prefill_tokens":0,"num_requests":4,"active_p_offloads":0,"cached_blocks":388,"cache_hit":0,"new_prefill":25923,"score_linear":191531.0,"score_lmetric":103692}],"chosen_score_linear":134156.0,"chosen_score_lmetric":3718,"t_first_token":348487.028505119,"t_first_token_unix":1779779352.042449,"t_done":348488.723673312,"t_done_unix":1779779353.7376165},{"request_id":"1286551:2:1289450:202","session_id":"1286551","input_length":9610,"t_proxy_recv":348488.305708819,"t_decision_unix":1779779353.3196504,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":1536,"affinity_cache_ratio":0.1598335067637877,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":false,"cache_hit":1536,"estimated_new_tokens":8074,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":488,"cache_hit":0,"new_prefill":9610,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":77805,"ongoing_decode_tokens":0,"pending_prefill_tokens":77805,"num_requests":1,"active_p_offloads":0,"cached_blocks":534,"cache_hit":0,"new_prefill":9610,"score_linear":77805.0,"score_lmetric":87415},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":596,"cache_hit":1536,"new_prefill":8074,"score_linear":-1536.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":673,"cache_hit":0,"new_prefill":9610,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":184766,"ongoing_decode_tokens":95826,"pending_prefill_tokens":876,"num_requests":3,"active_p_offloads":0,"cached_blocks":709,"cache_hit":0,"new_prefill":9610,"score_linear":184766.0,"score_lmetric":31458},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":443,"cache_hit":0,"new_prefill":9610,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":44920,"ongoing_decode_tokens":0,"pending_prefill_tokens":13688,"num_requests":1,"active_p_offloads":0,"cached_blocks":677,"cache_hit":0,"new_prefill":9610,"score_linear":44920.0,"score_lmetric":23298},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":191531,"ongoing_decode_tokens":191531,"pending_prefill_tokens":0,"num_requests":4,"active_p_offloads":0,"cached_blocks":388,"cache_hit":0,"new_prefill":9610,"score_linear":191531.0,"score_lmetric":38440}],"chosen_score_linear":-1536.0,"chosen_score_lmetric":0,"t_first_token":348488.927442943,"t_first_token_unix":1779779353.9413872,"t_done":348489.297706494,"t_done_unix":1779779354.3116493},{"request_id":"1287588:2:1289701:205","session_id":"1287588","input_length":10195,"t_proxy_recv":348489.217349869,"t_decision_unix":1779779354.2312913,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":9728,"affinity_cache_ratio":0.954193231976459,"affinity_num_requests":1,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":9728,"estimated_new_tokens":467,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":32191,"ongoing_decode_tokens":32191,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":488,"cache_hit":9728,"new_prefill":467,"score_linear":22463.0,"score_lmetric":467},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":77805,"ongoing_decode_tokens":0,"pending_prefill_tokens":77805,"num_requests":1,"active_p_offloads":0,"cached_blocks":534,"cache_hit":0,"new_prefill":10195,"score_linear":77805.0,"score_lmetric":88000},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":9610,"ongoing_decode_tokens":9610,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":596,"cache_hit":0,"new_prefill":10195,"score_linear":9610.0,"score_lmetric":10195},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":673,"cache_hit":0,"new_prefill":10195,"score_linear":82014.0,"score_lmetric":92209},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":158843,"ongoing_decode_tokens":158843,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":712,"cache_hit":0,"new_prefill":10195,"score_linear":158843.0,"score_lmetric":20390},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":443,"cache_hit":0,"new_prefill":10195,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":44920,"ongoing_decode_tokens":0,"pending_prefill_tokens":13688,"num_requests":1,"active_p_offloads":0,"cached_blocks":677,"cache_hit":0,"new_prefill":10195,"score_linear":44920.0,"score_lmetric":23883},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":191531,"ongoing_decode_tokens":191531,"pending_prefill_tokens":0,"num_requests":4,"active_p_offloads":0,"cached_blocks":388,"cache_hit":0,"new_prefill":10195,"score_linear":191531.0,"score_lmetric":40780}],"chosen_score_linear":22463.0,"chosen_score_lmetric":467,"t_first_token":348489.261338958,"t_first_token_unix":1779779354.2752821,"t_done":348489.35372655,"t_done_unix":1779779354.3676698},{"request_id":"1279412:4:1289482:203","session_id":"1279412","input_length":32191,"t_proxy_recv":348488.380577337,"t_decision_unix":1779779353.394519,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":31744,"affinity_cache_ratio":0.9861141312789289,"affinity_num_requests":0,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":31744,"estimated_new_tokens":447,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":488,"cache_hit":31744,"new_prefill":447,"score_linear":-31744.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":77805,"ongoing_decode_tokens":0,"pending_prefill_tokens":77805,"num_requests":1,"active_p_offloads":0,"cached_blocks":534,"cache_hit":0,"new_prefill":32191,"score_linear":77805.0,"score_lmetric":109996},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":9610,"ongoing_decode_tokens":0,"pending_prefill_tokens":8074,"num_requests":1,"active_p_offloads":0,"cached_blocks":596,"cache_hit":0,"new_prefill":32191,"score_linear":9610.0,"score_lmetric":40265},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":673,"cache_hit":0,"new_prefill":32191,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":184766,"ongoing_decode_tokens":95826,"pending_prefill_tokens":876,"num_requests":3,"active_p_offloads":0,"cached_blocks":709,"cache_hit":0,"new_prefill":32191,"score_linear":184766.0,"score_lmetric":99201},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":443,"cache_hit":0,"new_prefill":32191,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":44920,"ongoing_decode_tokens":0,"pending_prefill_tokens":13688,"num_requests":1,"active_p_offloads":0,"cached_blocks":677,"cache_hit":12800,"new_prefill":19391,"score_linear":32120.0,"score_lmetric":33079},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":191531,"ongoing_decode_tokens":191531,"pending_prefill_tokens":0,"num_requests":4,"active_p_offloads":0,"cached_blocks":388,"cache_hit":12800,"new_prefill":19391,"score_linear":178731.0,"score_lmetric":77564}],"chosen_score_linear":-31744.0,"chosen_score_lmetric":0,"t_first_token":348488.458515942,"t_first_token_unix":1779779353.4724596,"t_done":348489.525794002,"t_done_unix":1779779354.5397375},{"request_id":"1279994:2:1283011:177","session_id":"1279994","input_length":88940,"t_proxy_recv":348487.937907541,"t_decision_unix":1779779352.9518487,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":88064,"affinity_cache_ratio":0.990150663368563,"affinity_num_requests":2,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":88064,"estimated_new_tokens":876,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":488,"cache_hit":0,"new_prefill":88940,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":77805,"ongoing_decode_tokens":0,"pending_prefill_tokens":77805,"num_requests":1,"active_p_offloads":0,"cached_blocks":534,"cache_hit":0,"new_prefill":88940,"score_linear":77805.0,"score_lmetric":166745},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":596,"cache_hit":0,"new_prefill":88940,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":673,"cache_hit":0,"new_prefill":88940,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":95826,"ongoing_decode_tokens":95826,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":709,"cache_hit":88064,"new_prefill":876,"score_linear":7762.0,"score_lmetric":1752},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":443,"cache_hit":0,"new_prefill":88940,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":44920,"ongoing_decode_tokens":0,"pending_prefill_tokens":13688,"num_requests":1,"active_p_offloads":0,"cached_blocks":677,"cache_hit":0,"new_prefill":88940,"score_linear":44920.0,"score_lmetric":102628},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":191531,"ongoing_decode_tokens":191531,"pending_prefill_tokens":0,"num_requests":4,"active_p_offloads":0,"cached_blocks":388,"cache_hit":0,"new_prefill":88940,"score_linear":191531.0,"score_lmetric":355760}],"chosen_score_linear":7762.0,"chosen_score_lmetric":1752,"t_first_token":348488.391475293,"t_first_token_unix":1779779353.4054182,"t_done":348489.717944137,"t_done_unix":1779779354.7318876},{"request_id":"1268831:5:1284640:184","session_id":"1268831","input_length":42264,"t_proxy_recv":348471.76180844,"t_decision_unix":1779779336.77575,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":32256,"affinity_cache_ratio":0.7632027257240205,"affinity_num_requests":2,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":32256,"estimated_new_tokens":10008,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":441,"cache_hit":0,"new_prefill":42264,"score_linear":25187.0,"score_lmetric":42264},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":534,"cache_hit":0,"new_prefill":42264,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":582,"cache_hit":0,"new_prefill":42264,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":49619,"ongoing_decode_tokens":0,"pending_prefill_tokens":49619,"num_requests":1,"active_p_offloads":0,"cached_blocks":575,"cache_hit":0,"new_prefill":42264,"score_linear":49619.0,"score_lmetric":91883},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":140475,"ongoing_decode_tokens":27639,"pending_prefill_tokens":95940,"num_requests":2,"active_p_offloads":0,"cached_blocks":523,"cache_hit":32256,"new_prefill":10008,"score_linear":108219.0,"score_lmetric":211896},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":416,"cache_hit":0,"new_prefill":42264,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":30965,"ongoing_decode_tokens":30965,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":662,"cache_hit":0,"new_prefill":42264,"score_linear":30965.0,"score_lmetric":42264},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":388,"cache_hit":0,"new_prefill":42264,"score_linear":191531.0,"score_lmetric":701708}],"chosen_score_linear":108219.0,"chosen_score_lmetric":211896,"t_first_token":348485.860274633,"t_first_token_unix":1779779350.8742187,"t_done":348489.753149374,"t_done_unix":1779779354.7670922},{"request_id":"1272313:2:1275926:138","session_id":"1272313","input_length":10639,"t_proxy_recv":348441.71016759,"t_decision_unix":1779779306.7241087,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":9728,"affinity_cache_ratio":0.914371651471003,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":9728,"estimated_new_tokens":911,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":4096,"new_prefill":6543,"score_linear":131139.0,"score_lmetric":239524},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":10639,"score_linear":102074.0,"score_lmetric":109129},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":10639,"score_linear":150574.0,"score_lmetric":256890},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":108713,"ongoing_decode_tokens":15283,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":10639,"score_linear":108713.0,"score_lmetric":208138},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":447,"cache_hit":0,"new_prefill":10639,"score_linear":27639.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":314,"cache_hit":0,"new_prefill":10639,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":489,"cache_hit":0,"new_prefill":10639,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":114612,"ongoing_decode_tokens":0,"pending_prefill_tokens":114612,"num_requests":1,"active_p_offloads":0,"cached_blocks":388,"cache_hit":9728,"new_prefill":911,"score_linear":104884.0,"score_lmetric":115523}],"chosen_score_linear":104884.0,"chosen_score_lmetric":115523,"t_first_token":348483.450632349,"t_first_token_unix":1779779348.4645765,"t_done":348490.130636055,"t_done_unix":1779779355.1445794},{"request_id":"1272313:3:1279208:159","session_id":"1272313","input_length":11549,"t_proxy_recv":348490.134396808,"t_decision_unix":1779779355.1483383,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":7,"chosen_idx":0,"affinity_cache_hit":10752,"affinity_cache_ratio":0.9309896960775825,"affinity_num_requests":3,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":4096,"estimated_new_tokens":7453,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":489,"cache_hit":4096,"new_prefill":7453,"score_linear":-4096.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":77805,"ongoing_decode_tokens":0,"pending_prefill_tokens":77805,"num_requests":1,"active_p_offloads":0,"cached_blocks":534,"cache_hit":0,"new_prefill":11549,"score_linear":77805.0,"score_lmetric":89354},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":611,"cache_hit":0,"new_prefill":11549,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":673,"cache_hit":0,"new_prefill":11549,"score_linear":82014.0,"score_lmetric":93563},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":116865,"ongoing_decode_tokens":116865,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":732,"cache_hit":0,"new_prefill":11549,"score_linear":116865.0,"score_lmetric":11549},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":443,"cache_hit":0,"new_prefill":11549,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":44920,"ongoing_decode_tokens":0,"pending_prefill_tokens":13688,"num_requests":1,"active_p_offloads":0,"cached_blocks":677,"cache_hit":7680,"new_prefill":3869,"score_linear":37240.0,"score_lmetric":17557},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":180892,"ongoing_decode_tokens":180892,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":390,"cache_hit":10752,"new_prefill":797,"score_linear":170140.0,"score_lmetric":2391}],"chosen_score_linear":-4096.0,"chosen_score_lmetric":0,"t_first_token":348490.811207537,"t_first_token_unix":1779779355.8251512,"t_done":348491.994351795,"t_done_unix":1779779357.0082948},{"request_id":"1290442:1:1290442:210","session_id":"1290442","input_length":7261,"t_proxy_recv":348491.573027495,"t_decision_unix":1779779356.5869691,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":7261,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":7261,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":11549,"ongoing_decode_tokens":11549,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":489,"cache_hit":0,"new_prefill":7261,"score_linear":11549.0,"score_lmetric":7261},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":77805,"ongoing_decode_tokens":0,"pending_prefill_tokens":77805,"num_requests":1,"active_p_offloads":0,"cached_blocks":534,"cache_hit":0,"new_prefill":7261,"score_linear":77805.0,"score_lmetric":85066},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":13552,"ongoing_decode_tokens":0,"pending_prefill_tokens":12016,"num_requests":1,"active_p_offloads":0,"cached_blocks":611,"cache_hit":0,"new_prefill":7261,"score_linear":13552.0,"score_lmetric":19277},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":673,"cache_hit":0,"new_prefill":7261,"score_linear":82014.0,"score_lmetric":89275},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":116865,"ongoing_decode_tokens":116865,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":732,"cache_hit":0,"new_prefill":7261,"score_linear":116865.0,"score_lmetric":7261},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":23228,"ongoing_decode_tokens":0,"pending_prefill_tokens":23228,"num_requests":1,"active_p_offloads":0,"cached_blocks":443,"cache_hit":0,"new_prefill":7261,"score_linear":23228.0,"score_lmetric":30489},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":44920,"ongoing_decode_tokens":44920,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":677,"cache_hit":0,"new_prefill":7261,"score_linear":44920.0,"score_lmetric":7261},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":180892,"ongoing_decode_tokens":180892,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":390,"cache_hit":0,"new_prefill":7261,"score_linear":180892.0,"score_lmetric":21783}],"chosen_score_linear":44920.0,"chosen_score_lmetric":7261,"t_first_token":348492.074829058,"t_first_token_unix":1779779357.0887723,"t_done":348492.310448362,"t_done_unix":1779779357.3243918},{"request_id":"1290426:1:1290426:209","session_id":"1290426","input_length":13552,"t_proxy_recv":348491.548488405,"t_decision_unix":1779779356.5624297,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":false,"cache_hit":1536,"estimated_new_tokens":12016,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":11549,"ongoing_decode_tokens":11549,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":489,"cache_hit":0,"new_prefill":13552,"score_linear":11549.0,"score_lmetric":13552},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":77805,"ongoing_decode_tokens":0,"pending_prefill_tokens":77805,"num_requests":1,"active_p_offloads":0,"cached_blocks":534,"cache_hit":0,"new_prefill":13552,"score_linear":77805.0,"score_lmetric":91357},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":611,"cache_hit":1536,"new_prefill":12016,"score_linear":-1536.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":673,"cache_hit":0,"new_prefill":13552,"score_linear":82014.0,"score_lmetric":95566},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":116865,"ongoing_decode_tokens":116865,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":732,"cache_hit":0,"new_prefill":13552,"score_linear":116865.0,"score_lmetric":13552},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":23228,"ongoing_decode_tokens":0,"pending_prefill_tokens":23228,"num_requests":1,"active_p_offloads":0,"cached_blocks":443,"cache_hit":0,"new_prefill":13552,"score_linear":23228.0,"score_lmetric":36780},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":44920,"ongoing_decode_tokens":44920,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":677,"cache_hit":0,"new_prefill":13552,"score_linear":44920.0,"score_lmetric":13552},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":180892,"ongoing_decode_tokens":180892,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":390,"cache_hit":0,"new_prefill":13552,"score_linear":180892.0,"score_lmetric":40656}],"chosen_score_linear":-1536.0,"chosen_score_lmetric":0,"t_first_token":348492.634175814,"t_first_token_unix":1779779357.6481194,"t_done":348493.011410747,"t_done_unix":1779779358.0253544},{"request_id":"1272313:4:1281720:170","session_id":"1272313","input_length":13901,"t_proxy_recv":348491.999135548,"t_decision_unix":1779779357.0130773,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":11264,"affinity_cache_ratio":0.8103014171642328,"affinity_num_requests":0,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":11264,"estimated_new_tokens":2637,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":503,"cache_hit":11264,"new_prefill":2637,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":77805,"ongoing_decode_tokens":0,"pending_prefill_tokens":77805,"num_requests":1,"active_p_offloads":0,"cached_blocks":534,"cache_hit":0,"new_prefill":13901,"score_linear":77805.0,"score_lmetric":91706},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":13552,"ongoing_decode_tokens":0,"pending_prefill_tokens":12016,"num_requests":1,"active_p_offloads":0,"cached_blocks":611,"cache_hit":0,"new_prefill":13901,"score_linear":13552.0,"score_lmetric":25917},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":673,"cache_hit":0,"new_prefill":13901,"score_linear":82014.0,"score_lmetric":95915},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":116865,"ongoing_decode_tokens":116865,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":732,"cache_hit":0,"new_prefill":13901,"score_linear":116865.0,"score_lmetric":13901},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":23228,"ongoing_decode_tokens":0,"pending_prefill_tokens":23228,"num_requests":1,"active_p_offloads":0,"cached_blocks":443,"cache_hit":0,"new_prefill":13901,"score_linear":23228.0,"score_lmetric":37129},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":52181,"ongoing_decode_tokens":44920,"pending_prefill_tokens":7261,"num_requests":2,"active_p_offloads":0,"cached_blocks":677,"cache_hit":7680,"new_prefill":6221,"score_linear":44501.0,"score_lmetric":26964},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":180892,"ongoing_decode_tokens":180892,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":390,"cache_hit":10752,"new_prefill":3149,"score_linear":170140.0,"score_lmetric":9447}],"chosen_score_linear":-11264.0,"chosen_score_lmetric":0,"t_first_token":348492.334942578,"t_first_token_unix":1779779357.3488863,"t_done":348493.909533937,"t_done_unix":1779779358.923477},{"request_id":"1291100:1:1291100:212","session_id":"1291100","input_length":1744,"t_proxy_recv":348493.787717952,"t_decision_unix":1779779358.801659,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":1744,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":13901,"ongoing_decode_tokens":13901,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":503,"cache_hit":0,"new_prefill":1744,"score_linear":13901.0,"score_lmetric":1744},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":77805,"ongoing_decode_tokens":0,"pending_prefill_tokens":77805,"num_requests":1,"active_p_offloads":0,"cached_blocks":534,"cache_hit":0,"new_prefill":1744,"score_linear":77805.0,"score_lmetric":79549},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":620,"cache_hit":0,"new_prefill":1744,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":673,"cache_hit":0,"new_prefill":1744,"score_linear":82014.0,"score_lmetric":83758},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":116865,"ongoing_decode_tokens":116865,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":732,"cache_hit":0,"new_prefill":1744,"score_linear":116865.0,"score_lmetric":1744},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":23228,"ongoing_decode_tokens":0,"pending_prefill_tokens":23228,"num_requests":1,"active_p_offloads":0,"cached_blocks":443,"cache_hit":0,"new_prefill":1744,"score_linear":23228.0,"score_lmetric":24972},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":44920,"ongoing_decode_tokens":44920,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":691,"cache_hit":0,"new_prefill":1744,"score_linear":44920.0,"score_lmetric":1744},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":180892,"ongoing_decode_tokens":180892,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":390,"cache_hit":0,"new_prefill":1744,"score_linear":180892.0,"score_lmetric":5232}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348493.902113858,"t_first_token_unix":1779779358.916057,"t_done":348494.299468691,"t_done_unix":1779779359.3134122},{"request_id":"1290365:1:1290365:208","session_id":"1290365","input_length":23228,"t_proxy_recv":348491.372841182,"t_decision_unix":1779779356.3867822,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":23228,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":11549,"ongoing_decode_tokens":11549,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":489,"cache_hit":0,"new_prefill":23228,"score_linear":11549.0,"score_lmetric":23228},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":77805,"ongoing_decode_tokens":0,"pending_prefill_tokens":77805,"num_requests":1,"active_p_offloads":0,"cached_blocks":534,"cache_hit":0,"new_prefill":23228,"score_linear":77805.0,"score_lmetric":101033},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":611,"cache_hit":0,"new_prefill":23228,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":673,"cache_hit":0,"new_prefill":23228,"score_linear":82014.0,"score_lmetric":105242},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":116865,"ongoing_decode_tokens":116865,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":732,"cache_hit":0,"new_prefill":23228,"score_linear":116865.0,"score_lmetric":23228},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":443,"cache_hit":0,"new_prefill":23228,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":44920,"ongoing_decode_tokens":44920,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":677,"cache_hit":0,"new_prefill":23228,"score_linear":44920.0,"score_lmetric":23228},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":180892,"ongoing_decode_tokens":180892,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":390,"cache_hit":0,"new_prefill":23228,"score_linear":180892.0,"score_lmetric":69684}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348493.943462729,"t_first_token_unix":1779779358.9574058,"t_done":348494.674589651,"t_done_unix":1779779359.6885333},{"request_id":"1266668:4:1276519:143","session_id":"1266668","input_length":49380,"t_proxy_recv":348443.783072602,"t_decision_unix":1779779308.7970135,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":33792,"affinity_cache_ratio":0.684325637910085,"affinity_num_requests":2,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":33792,"estimated_new_tokens":15588,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":371,"cache_hit":4096,"new_prefill":45284,"score_linear":21091.0,"score_lmetric":45284},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":49380,"score_linear":102074.0,"score_lmetric":147870},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":49380,"score_linear":150574.0,"score_lmetric":334372},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":108713,"ongoing_decode_tokens":15283,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":49380,"score_linear":108713.0,"score_lmetric":285620},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":44919,"ongoing_decode_tokens":27639,"pending_prefill_tokens":17280,"num_requests":1,"active_p_offloads":0,"cached_blocks":447,"cache_hit":0,"new_prefill":49380,"score_linear":44919.0,"score_lmetric":66660},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":15783,"ongoing_decode_tokens":0,"pending_prefill_tokens":15783,"num_requests":1,"active_p_offloads":0,"cached_blocks":314,"cache_hit":0,"new_prefill":49380,"score_linear":15783.0,"score_lmetric":65163},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":8704,"ongoing_decode_tokens":0,"pending_prefill_tokens":8704,"num_requests":1,"active_p_offloads":0,"cached_blocks":489,"cache_hit":0,"new_prefill":49380,"score_linear":8704.0,"score_lmetric":58084},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":125251,"ongoing_decode_tokens":0,"pending_prefill_tokens":115523,"num_requests":2,"active_p_offloads":0,"cached_blocks":388,"cache_hit":33792,"new_prefill":15588,"score_linear":91459.0,"score_lmetric":262222}],"chosen_score_linear":91459.0,"chosen_score_lmetric":262222,"t_first_token":348485.88266802,"t_first_token_unix":1779779350.8966117,"t_done":348495.048288857,"t_done_unix":1779779360.0622323},{"request_id":"1279994:3:1286181:192","session_id":"1279994","input_length":89226,"t_proxy_recv":348489.743356795,"t_decision_unix":1779779354.757298,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":88576,"affinity_cache_ratio":0.9927151278775245,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":88576,"estimated_new_tokens":650,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":489,"cache_hit":0,"new_prefill":89226,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":77805,"ongoing_decode_tokens":0,"pending_prefill_tokens":77805,"num_requests":1,"active_p_offloads":0,"cached_blocks":534,"cache_hit":0,"new_prefill":89226,"score_linear":77805.0,"score_lmetric":167031},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":611,"cache_hit":0,"new_prefill":89226,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":673,"cache_hit":0,"new_prefill":89226,"score_linear":82014.0,"score_lmetric":171240},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":69903,"ongoing_decode_tokens":69903,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":713,"cache_hit":88576,"new_prefill":650,"score_linear":-18673.0,"score_lmetric":650},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":443,"cache_hit":0,"new_prefill":89226,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":44920,"ongoing_decode_tokens":0,"pending_prefill_tokens":13688,"num_requests":1,"active_p_offloads":0,"cached_blocks":677,"cache_hit":0,"new_prefill":89226,"score_linear":44920.0,"score_lmetric":102914},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":191531,"ongoing_decode_tokens":191531,"pending_prefill_tokens":0,"num_requests":4,"active_p_offloads":0,"cached_blocks":388,"cache_hit":0,"new_prefill":89226,"score_linear":191531.0,"score_lmetric":356904}],"chosen_score_linear":-18673.0,"chosen_score_lmetric":650,"t_first_token":348490.024101339,"t_first_token_unix":1779779355.0380452,"t_done":348495.957877587,"t_done_unix":1779779360.971821},{"request_id":"1262354:12:1291548:213","session_id":"1262354","input_length":35099,"t_proxy_recv":348495.427777252,"t_decision_unix":1779779360.4417183,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":34816,"affinity_cache_ratio":0.9919370922248497,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":34816,"estimated_new_tokens":283,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":14885,"ongoing_decode_tokens":14885,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":508,"cache_hit":12800,"new_prefill":22299,"score_linear":2085.0,"score_lmetric":22299},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":77805,"ongoing_decode_tokens":0,"pending_prefill_tokens":77805,"num_requests":1,"active_p_offloads":0,"cached_blocks":534,"cache_hit":0,"new_prefill":35099,"score_linear":77805.0,"score_lmetric":112904},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":623,"cache_hit":0,"new_prefill":35099,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":673,"cache_hit":0,"new_prefill":35099,"score_linear":82014.0,"score_lmetric":117113},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":116865,"ongoing_decode_tokens":116865,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":732,"cache_hit":0,"new_prefill":35099,"score_linear":116865.0,"score_lmetric":35099},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":484,"cache_hit":0,"new_prefill":35099,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":44920,"ongoing_decode_tokens":44920,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":691,"cache_hit":34816,"new_prefill":283,"score_linear":10104.0,"score_lmetric":283},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":189544,"ongoing_decode_tokens":131512,"pending_prefill_tokens":8368,"num_requests":3,"active_p_offloads":0,"cached_blocks":421,"cache_hit":28672,"new_prefill":6427,"score_linear":160872.0,"score_lmetric":44385}],"chosen_score_linear":10104.0,"chosen_score_lmetric":283,"t_first_token":348495.510079679,"t_first_token_unix":1779779360.5240226,"t_done":348496.284029016,"t_done_unix":1779779361.2979722},{"request_id":"1279412:5:1291651:214","session_id":"1279412","input_length":32359,"t_proxy_recv":348495.712014849,"t_decision_unix":1779779360.7259562,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":32256,"affinity_cache_ratio":0.9968169597329954,"affinity_num_requests":1,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":32256,"estimated_new_tokens":103,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":14885,"ongoing_decode_tokens":14885,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":508,"cache_hit":32256,"new_prefill":103,"score_linear":-17371.0,"score_lmetric":103},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":77805,"ongoing_decode_tokens":0,"pending_prefill_tokens":77805,"num_requests":1,"active_p_offloads":0,"cached_blocks":534,"cache_hit":0,"new_prefill":32359,"score_linear":77805.0,"score_lmetric":110164},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":623,"cache_hit":0,"new_prefill":32359,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":673,"cache_hit":0,"new_prefill":32359,"score_linear":82014.0,"score_lmetric":114373},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":116865,"ongoing_decode_tokens":116865,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":732,"cache_hit":0,"new_prefill":32359,"score_linear":116865.0,"score_lmetric":32359},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":484,"cache_hit":0,"new_prefill":32359,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80019,"ongoing_decode_tokens":80019,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":691,"cache_hit":12800,"new_prefill":19559,"score_linear":67219.0,"score_lmetric":39118},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":189544,"ongoing_decode_tokens":131512,"pending_prefill_tokens":8368,"num_requests":3,"active_p_offloads":0,"cached_blocks":421,"cache_hit":12800,"new_prefill":19559,"score_linear":176744.0,"score_lmetric":83781}],"chosen_score_linear":-17371.0,"chosen_score_lmetric":103,"t_first_token":348495.793093174,"t_first_token_unix":1779779360.8070364,"t_done":348496.518905484,"t_done_unix":1779779361.5328484},{"request_id":"1272313:5:1284744:185","session_id":"1272313","input_length":14885,"t_proxy_recv":348493.917507032,"t_decision_unix":1779779358.9314485,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":13824,"affinity_cache_ratio":0.9287201881088344,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":13824,"estimated_new_tokens":1061,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":508,"cache_hit":13824,"new_prefill":1061,"score_linear":-13824.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":77805,"ongoing_decode_tokens":0,"pending_prefill_tokens":77805,"num_requests":1,"active_p_offloads":0,"cached_blocks":534,"cache_hit":0,"new_prefill":14885,"score_linear":77805.0,"score_lmetric":92690},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":1744,"ongoing_decode_tokens":1744,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":620,"cache_hit":0,"new_prefill":14885,"score_linear":1744.0,"score_lmetric":14885},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":673,"cache_hit":0,"new_prefill":14885,"score_linear":82014.0,"score_lmetric":96899},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":116865,"ongoing_decode_tokens":116865,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":732,"cache_hit":0,"new_prefill":14885,"score_linear":116865.0,"score_lmetric":14885},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":23228,"ongoing_decode_tokens":0,"pending_prefill_tokens":23228,"num_requests":1,"active_p_offloads":0,"cached_blocks":443,"cache_hit":0,"new_prefill":14885,"score_linear":23228.0,"score_lmetric":38113},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":44920,"ongoing_decode_tokens":44920,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":691,"cache_hit":7680,"new_prefill":7205,"score_linear":37240.0,"score_lmetric":7205},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":180892,"ongoing_decode_tokens":180892,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":390,"cache_hit":10752,"new_prefill":4133,"score_linear":170140.0,"score_lmetric":12399}],"chosen_score_linear":-13824.0,"chosen_score_lmetric":0,"t_first_token":348494.071857098,"t_first_token_unix":1779779359.0858004,"t_done":348497.155417119,"t_done_unix":1779779362.1693606},{"request_id":"1260327:6:1291827:215","session_id":"1260327","input_length":42153,"t_proxy_recv":348496.396225197,"t_decision_unix":1779779361.4101667,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":41984,"affinity_cache_ratio":0.9959907954356748,"affinity_num_requests":1,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":41984,"estimated_new_tokens":169,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":47244,"ongoing_decode_tokens":47244,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":508,"cache_hit":0,"new_prefill":42153,"score_linear":47244.0,"score_lmetric":84306},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":77805,"ongoing_decode_tokens":0,"pending_prefill_tokens":77805,"num_requests":1,"active_p_offloads":0,"cached_blocks":534,"cache_hit":0,"new_prefill":42153,"score_linear":77805.0,"score_lmetric":119958},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":623,"cache_hit":0,"new_prefill":42153,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":673,"cache_hit":0,"new_prefill":42153,"score_linear":82014.0,"score_lmetric":124167},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":117388,"ongoing_decode_tokens":117388,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":734,"cache_hit":0,"new_prefill":42153,"score_linear":117388.0,"score_lmetric":42153},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":484,"cache_hit":0,"new_prefill":42153,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":44920,"ongoing_decode_tokens":44920,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":691,"cache_hit":41984,"new_prefill":169,"score_linear":2936.0,"score_lmetric":169},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":189544,"ongoing_decode_tokens":131512,"pending_prefill_tokens":8368,"num_requests":3,"active_p_offloads":0,"cached_blocks":421,"cache_hit":31232,"new_prefill":10921,"score_linear":158312.0,"score_lmetric":57867}],"chosen_score_linear":2936.0,"chosen_score_lmetric":169,"t_first_token":348496.492365427,"t_first_token_unix":1779779361.5063083,"t_done":348497.162893036,"t_done_unix":1779779362.1768363},{"request_id":"1291944:1:1291944:216","session_id":"1291944","input_length":8144,"t_proxy_recv":348496.766929353,"t_decision_unix":1779779361.7808707,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":8144,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":14885,"ongoing_decode_tokens":14885,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":508,"cache_hit":0,"new_prefill":8144,"score_linear":14885.0,"score_lmetric":8144},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":77805,"ongoing_decode_tokens":0,"pending_prefill_tokens":77805,"num_requests":1,"active_p_offloads":0,"cached_blocks":534,"cache_hit":0,"new_prefill":8144,"score_linear":77805.0,"score_lmetric":85949},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":623,"cache_hit":0,"new_prefill":8144,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":673,"cache_hit":0,"new_prefill":8144,"score_linear":82014.0,"score_lmetric":90158},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":117388,"ongoing_decode_tokens":117388,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":734,"cache_hit":0,"new_prefill":8144,"score_linear":117388.0,"score_lmetric":8144},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":484,"cache_hit":0,"new_prefill":8144,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":87073,"ongoing_decode_tokens":87073,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":691,"cache_hit":0,"new_prefill":8144,"score_linear":87073.0,"score_lmetric":16288},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":189544,"ongoing_decode_tokens":131512,"pending_prefill_tokens":8368,"num_requests":3,"active_p_offloads":0,"cached_blocks":421,"cache_hit":0,"new_prefill":8144,"score_linear":189544.0,"score_lmetric":49536}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348497.340710772,"t_first_token_unix":1779779362.3546548,"t_done":348497.568916446,"t_done_unix":1779779362.5828595},{"request_id":"1290365:2:1292000:217","session_id":"1290365","input_length":23414,"t_proxy_recv":348496.963211861,"t_decision_unix":1779779361.9771528,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":23040,"affinity_cache_ratio":0.9840266507217904,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":23040,"estimated_new_tokens":374,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":14885,"ongoing_decode_tokens":14885,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":508,"cache_hit":0,"new_prefill":23414,"score_linear":14885.0,"score_lmetric":23414},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":77805,"ongoing_decode_tokens":0,"pending_prefill_tokens":77805,"num_requests":1,"active_p_offloads":0,"cached_blocks":534,"cache_hit":0,"new_prefill":23414,"score_linear":77805.0,"score_lmetric":101219},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":623,"cache_hit":0,"new_prefill":23414,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":673,"cache_hit":0,"new_prefill":23414,"score_linear":82014.0,"score_lmetric":105428},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":117388,"ongoing_decode_tokens":117388,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":734,"cache_hit":0,"new_prefill":23414,"score_linear":117388.0,"score_lmetric":23414},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":8144,"ongoing_decode_tokens":0,"pending_prefill_tokens":8144,"num_requests":1,"active_p_offloads":0,"cached_blocks":484,"cache_hit":23040,"new_prefill":374,"score_linear":-14896.0,"score_lmetric":8518},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":87073,"ongoing_decode_tokens":87073,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":691,"cache_hit":0,"new_prefill":23414,"score_linear":87073.0,"score_lmetric":46828},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":189544,"ongoing_decode_tokens":131512,"pending_prefill_tokens":8368,"num_requests":3,"active_p_offloads":0,"cached_blocks":421,"cache_hit":0,"new_prefill":23414,"score_linear":189544.0,"score_lmetric":95346}],"chosen_score_linear":-14896.0,"chosen_score_lmetric":8518,"t_first_token":348497.386911526,"t_first_token_unix":1779779362.4008548,"t_done":348497.738992218,"t_done_unix":1779779362.7529352},{"request_id":"1279994:4:1290711:211","session_id":"1279994","input_length":89749,"t_proxy_recv":348495.974816264,"t_decision_unix":1779779360.9887578,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":89600,"affinity_cache_ratio":0.9983398143711908,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":89600,"estimated_new_tokens":149,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":47244,"ongoing_decode_tokens":47244,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":508,"cache_hit":0,"new_prefill":89749,"score_linear":47244.0,"score_lmetric":179498},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":77805,"ongoing_decode_tokens":0,"pending_prefill_tokens":77805,"num_requests":1,"active_p_offloads":0,"cached_blocks":534,"cache_hit":0,"new_prefill":89749,"score_linear":77805.0,"score_lmetric":167554},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":623,"cache_hit":0,"new_prefill":89749,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":673,"cache_hit":0,"new_prefill":89749,"score_linear":82014.0,"score_lmetric":171763},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":734,"cache_hit":89600,"new_prefill":149,"score_linear":-61961.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":484,"cache_hit":0,"new_prefill":89749,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80019,"ongoing_decode_tokens":80019,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":691,"cache_hit":0,"new_prefill":89749,"score_linear":80019.0,"score_lmetric":179498},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":189544,"ongoing_decode_tokens":131512,"pending_prefill_tokens":8368,"num_requests":3,"active_p_offloads":0,"cached_blocks":421,"cache_hit":0,"new_prefill":89749,"score_linear":189544.0,"score_lmetric":294351}],"chosen_score_linear":-61961.0,"chosen_score_lmetric":0,"t_first_token":348496.148799733,"t_first_token_unix":1779779361.1627436,"t_done":348498.704438883,"t_done_unix":1779779363.7183821},{"request_id":"1272313:6:1287745:199","session_id":"1272313","input_length":15547,"t_proxy_recv":348497.159635443,"t_decision_unix":1779779362.1735773,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":15360,"affinity_cache_ratio":0.9879719560043738,"affinity_num_requests":0,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":15360,"estimated_new_tokens":187,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":511,"cache_hit":15360,"new_prefill":187,"score_linear":-15360.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":77805,"ongoing_decode_tokens":0,"pending_prefill_tokens":77805,"num_requests":1,"active_p_offloads":0,"cached_blocks":534,"cache_hit":0,"new_prefill":15547,"score_linear":77805.0,"score_lmetric":93352},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":623,"cache_hit":0,"new_prefill":15547,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":673,"cache_hit":0,"new_prefill":15547,"score_linear":82014.0,"score_lmetric":97561},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":117388,"ongoing_decode_tokens":117388,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":734,"cache_hit":0,"new_prefill":15547,"score_linear":117388.0,"score_lmetric":15547},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":31558,"ongoing_decode_tokens":0,"pending_prefill_tokens":8518,"num_requests":2,"active_p_offloads":0,"cached_blocks":484,"cache_hit":0,"new_prefill":15547,"score_linear":31558.0,"score_lmetric":48130},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":87073,"ongoing_decode_tokens":87073,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":691,"cache_hit":7680,"new_prefill":7867,"score_linear":79393.0,"score_lmetric":15734},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":189544,"ongoing_decode_tokens":131512,"pending_prefill_tokens":8368,"num_requests":3,"active_p_offloads":0,"cached_blocks":421,"cache_hit":10752,"new_prefill":4795,"score_linear":178792.0,"score_lmetric":39489}],"chosen_score_linear":-15360.0,"chosen_score_lmetric":0,"t_first_token":348497.234781997,"t_first_token_unix":1779779362.2487252,"t_done":348498.809959712,"t_done_unix":1779779363.8239026},{"request_id":"1269373:3:1277365:148","session_id":"1269373","input_length":16900,"t_proxy_recv":348446.814990573,"t_decision_unix":1779779311.828932,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":14848,"affinity_cache_ratio":0.8785798816568048,"affinity_num_requests":3,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":14848,"estimated_new_tokens":2052,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25187,"ongoing_decode_tokens":25187,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":371,"cache_hit":4096,"new_prefill":12804,"score_linear":21091.0,"score_lmetric":12804},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":16900,"score_linear":102074.0,"score_lmetric":115390},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":16900,"score_linear":150574.0,"score_lmetric":269412},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":108713,"ongoing_decode_tokens":15283,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":16900,"score_linear":108713.0,"score_lmetric":220660},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":70776,"ongoing_decode_tokens":38931,"pending_prefill_tokens":12389,"num_requests":2,"active_p_offloads":0,"cached_blocks":480,"cache_hit":0,"new_prefill":16900,"score_linear":70776.0,"score_lmetric":58578},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":344,"cache_hit":0,"new_prefill":16900,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":33524,"ongoing_decode_tokens":33524,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":506,"cache_hit":0,"new_prefill":16900,"score_linear":33524.0,"score_lmetric":16900},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":174631,"ongoing_decode_tokens":0,"pending_prefill_tokens":131111,"num_requests":3,"active_p_offloads":0,"cached_blocks":388,"cache_hit":14848,"new_prefill":2052,"score_linear":159783.0,"score_lmetric":399489}],"chosen_score_linear":159783.0,"chosen_score_lmetric":399489,"t_first_token":348486.076655885,"t_first_token_unix":1779779351.090599,"t_done":348499.173298962,"t_done_unix":1779779364.1872427},{"request_id":"1286551:3:1292589:219","session_id":"1286551","input_length":16914,"t_proxy_recv":348498.634254959,"t_decision_unix":1779779363.6481965,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":9216,"affinity_cache_ratio":0.54487406881873,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":9216,"estimated_new_tokens":7698,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":15547,"ongoing_decode_tokens":15547,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":511,"cache_hit":0,"new_prefill":16914,"score_linear":15547.0,"score_lmetric":16914},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":77805,"ongoing_decode_tokens":0,"pending_prefill_tokens":77805,"num_requests":1,"active_p_offloads":0,"cached_blocks":534,"cache_hit":0,"new_prefill":16914,"score_linear":77805.0,"score_lmetric":94719},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":623,"cache_hit":9216,"new_prefill":7698,"score_linear":-9216.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":673,"cache_hit":0,"new_prefill":16914,"score_linear":82014.0,"score_lmetric":98928},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":117388,"ongoing_decode_tokens":117388,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":734,"cache_hit":0,"new_prefill":16914,"score_linear":117388.0,"score_lmetric":16914},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":499,"cache_hit":0,"new_prefill":16914,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":44920,"ongoing_decode_tokens":44920,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":691,"cache_hit":0,"new_prefill":16914,"score_linear":44920.0,"score_lmetric":16914},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":189544,"ongoing_decode_tokens":189544,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":421,"cache_hit":0,"new_prefill":16914,"score_linear":189544.0,"score_lmetric":50742}],"chosen_score_linear":-9216.0,"chosen_score_lmetric":0,"t_first_token":348499.527270684,"t_first_token_unix":1779779364.5412138,"t_done":348499.711855492,"t_done_unix":1779779364.7257984},{"request_id":"1292750:1:1292750:220","session_id":"1292750","input_length":4684,"t_proxy_recv":348499.271132124,"t_decision_unix":1779779364.2850738,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":4684,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":15969,"ongoing_decode_tokens":15969,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":511,"cache_hit":0,"new_prefill":4684,"score_linear":15969.0,"score_lmetric":4684},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":77805,"ongoing_decode_tokens":0,"pending_prefill_tokens":77805,"num_requests":1,"active_p_offloads":0,"cached_blocks":534,"cache_hit":0,"new_prefill":4684,"score_linear":77805.0,"score_lmetric":82489},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":16914,"ongoing_decode_tokens":0,"pending_prefill_tokens":7698,"num_requests":1,"active_p_offloads":0,"cached_blocks":623,"cache_hit":0,"new_prefill":4684,"score_linear":16914.0,"score_lmetric":12382},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":673,"cache_hit":0,"new_prefill":4684,"score_linear":82014.0,"score_lmetric":86698},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":734,"cache_hit":0,"new_prefill":4684,"score_linear":27639.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":499,"cache_hit":0,"new_prefill":4684,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":44920,"ongoing_decode_tokens":44920,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":691,"cache_hit":0,"new_prefill":4684,"score_linear":44920.0,"score_lmetric":4684},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":202906,"ongoing_decode_tokens":172644,"pending_prefill_tokens":12854,"num_requests":3,"active_p_offloads":0,"cached_blocks":426,"cache_hit":0,"new_prefill":4684,"score_linear":202906.0,"score_lmetric":52614}],"chosen_score_linear":27639.0,"chosen_score_lmetric":0,"t_first_token":348499.574452644,"t_first_token_unix":1779779364.5883958,"t_done":348500.239348997,"t_done_unix":1779779365.253292},{"request_id":"1272313:7:1290281:206","session_id":"1272313","input_length":15969,"t_proxy_recv":348498.814644882,"t_decision_unix":1779779363.8285863,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":15360,"affinity_cache_ratio":0.96186361074582,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":15360,"estimated_new_tokens":609,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":511,"cache_hit":15360,"new_prefill":609,"score_linear":-15360.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":77805,"ongoing_decode_tokens":0,"pending_prefill_tokens":77805,"num_requests":1,"active_p_offloads":0,"cached_blocks":534,"cache_hit":0,"new_prefill":15969,"score_linear":77805.0,"score_lmetric":93774},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":16914,"ongoing_decode_tokens":0,"pending_prefill_tokens":7698,"num_requests":1,"active_p_offloads":0,"cached_blocks":623,"cache_hit":0,"new_prefill":15969,"score_linear":16914.0,"score_lmetric":23667},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":673,"cache_hit":0,"new_prefill":15969,"score_linear":82014.0,"score_lmetric":97983},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":734,"cache_hit":0,"new_prefill":15969,"score_linear":27639.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":499,"cache_hit":0,"new_prefill":15969,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":44920,"ongoing_decode_tokens":44920,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":691,"cache_hit":7680,"new_prefill":8289,"score_linear":37240.0,"score_lmetric":8289},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":189544,"ongoing_decode_tokens":189544,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":421,"cache_hit":10752,"new_prefill":5217,"score_linear":178792.0,"score_lmetric":15651}],"chosen_score_linear":-15360.0,"chosen_score_lmetric":0,"t_first_token":348498.887514139,"t_first_token_unix":1779779363.901457,"t_done":348500.88277833,"t_done_unix":1779779365.8967216},{"request_id":"1262354:13:1293000:221","session_id":"1262354","input_length":35210,"t_proxy_recv":348500.114128263,"t_decision_unix":1779779365.1280699,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":34816,"affinity_cache_ratio":0.9888099971598977,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":34816,"estimated_new_tokens":394,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":15969,"ongoing_decode_tokens":15969,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":511,"cache_hit":12800,"new_prefill":22410,"score_linear":3169.0,"score_lmetric":22410},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":77805,"ongoing_decode_tokens":0,"pending_prefill_tokens":77805,"num_requests":1,"active_p_offloads":0,"cached_blocks":534,"cache_hit":0,"new_prefill":35210,"score_linear":77805.0,"score_lmetric":113015},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":631,"cache_hit":0,"new_prefill":35210,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":673,"cache_hit":0,"new_prefill":35210,"score_linear":82014.0,"score_lmetric":117224},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":32323,"ongoing_decode_tokens":32323,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":734,"cache_hit":0,"new_prefill":35210,"score_linear":32323.0,"score_lmetric":35210},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":499,"cache_hit":0,"new_prefill":35210,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":44920,"ongoing_decode_tokens":44920,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":691,"cache_hit":34816,"new_prefill":394,"score_linear":10104.0,"score_lmetric":394},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":202906,"ongoing_decode_tokens":172644,"pending_prefill_tokens":12854,"num_requests":3,"active_p_offloads":0,"cached_blocks":426,"cache_hit":28672,"new_prefill":6538,"score_linear":174234.0,"score_lmetric":58176}],"chosen_score_linear":10104.0,"chosen_score_lmetric":394,"t_first_token":348500.195673592,"t_first_token_unix":1779779365.2096198,"t_done":348501.079685347,"t_done_unix":1779779366.0936286},{"request_id":"1287100:1:1287100:196","session_id":"1287100","input_length":77805,"t_proxy_recv":348480.423719996,"t_decision_unix":1779779345.4376552,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":77805,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":31898,"ongoing_decode_tokens":31898,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":459,"cache_hit":0,"new_prefill":77805,"score_linear":31898.0,"score_lmetric":77805},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":534,"cache_hit":0,"new_prefill":77805,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":596,"cache_hit":0,"new_prefill":77805,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":49619,"ongoing_decode_tokens":49619,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":575,"cache_hit":0,"new_prefill":77805,"score_linear":49619.0,"score_lmetric":77805},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":182739,"ongoing_decode_tokens":27639,"pending_prefill_tokens":105948,"num_requests":3,"active_p_offloads":0,"cached_blocks":523,"cache_hit":0,"new_prefill":77805,"score_linear":182739.0,"score_lmetric":551259},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":13962,"ongoing_decode_tokens":0,"pending_prefill_tokens":13962,"num_requests":1,"active_p_offloads":0,"cached_blocks":416,"cache_hit":0,"new_prefill":77805,"score_linear":13962.0,"score_lmetric":91767},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":676,"cache_hit":0,"new_prefill":77805,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":191531,"ongoing_decode_tokens":0,"pending_prefill_tokens":133163,"num_requests":4,"active_p_offloads":0,"cached_blocks":388,"cache_hit":0,"new_prefill":77805,"score_linear":191531.0,"score_lmetric":843872}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348501.121826695,"t_first_token_unix":1779779366.1357706,"t_done":348501.662337401,"t_done_unix":1779779366.6762815},{"request_id":"1291100:2:1293458:223","session_id":"1291100","input_length":2572,"t_proxy_recv":348501.757913899,"t_decision_unix":1779779366.7718554,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":1536,"affinity_cache_ratio":0.5972006220839814,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":1536,"estimated_new_tokens":1036,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":512,"cache_hit":0,"new_prefill":2572,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":686,"cache_hit":0,"new_prefill":2572,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":631,"cache_hit":1536,"new_prefill":1036,"score_linear":-1536.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":673,"cache_hit":0,"new_prefill":2572,"score_linear":82014.0,"score_lmetric":84586},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":743,"cache_hit":0,"new_prefill":2572,"score_linear":27639.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":24482,"ongoing_decode_tokens":24482,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":499,"cache_hit":0,"new_prefill":2572,"score_linear":24482.0,"score_lmetric":2572},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":44920,"ongoing_decode_tokens":44920,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":691,"cache_hit":0,"new_prefill":2572,"score_linear":44920.0,"score_lmetric":2572},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":202906,"ongoing_decode_tokens":202906,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":426,"cache_hit":0,"new_prefill":2572,"score_linear":202906.0,"score_lmetric":7716}],"chosen_score_linear":-1536.0,"chosen_score_lmetric":0,"t_first_token":348501.833272057,"t_first_token_unix":1779779366.8472154,"t_done":348501.98530625,"t_done_unix":1779779366.9992492},{"request_id":"1293507:1:1293507:224","session_id":"1293507","input_length":6460,"t_proxy_recv":348501.944077336,"t_decision_unix":1779779366.9580188,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":6460,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":512,"cache_hit":0,"new_prefill":6460,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":686,"cache_hit":0,"new_prefill":6460,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":2572,"ongoing_decode_tokens":2572,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":631,"cache_hit":0,"new_prefill":6460,"score_linear":2572.0,"score_lmetric":6460},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":673,"cache_hit":0,"new_prefill":6460,"score_linear":82014.0,"score_lmetric":88474},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":743,"cache_hit":0,"new_prefill":6460,"score_linear":27639.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":24482,"ongoing_decode_tokens":24482,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":499,"cache_hit":0,"new_prefill":6460,"score_linear":24482.0,"score_lmetric":6460},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":44920,"ongoing_decode_tokens":44920,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":691,"cache_hit":0,"new_prefill":6460,"score_linear":44920.0,"score_lmetric":6460},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":202906,"ongoing_decode_tokens":202906,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":426,"cache_hit":0,"new_prefill":6460,"score_linear":202906.0,"score_lmetric":19380}],"chosen_score_linear":27639.0,"chosen_score_lmetric":0,"t_first_token":348502.381222145,"t_first_token_unix":1779779367.3951657,"t_done":348502.572436791,"t_done_unix":1779779367.5863798},{"request_id":"1239034:4:1263738:78","session_id":"1239034","input_length":27639,"t_proxy_recv":348405.052616652,"t_decision_unix":1779779270.0665576,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":22528,"affinity_cache_ratio":0.8150801403813452,"affinity_num_requests":-1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":22528,"estimated_new_tokens":5111,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":110048,"ongoing_decode_tokens":0,"pending_prefill_tokens":110048,"num_requests":1,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":27639,"score_linear":110048.0,"score_lmetric":137687},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55989,"ongoing_decode_tokens":55989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":261,"cache_hit":0,"new_prefill":27639,"score_linear":55989.0,"score_lmetric":27639},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":14329,"ongoing_decode_tokens":301,"pending_prefill_tokens":14028,"num_requests":2,"active_p_offloads":0,"cached_blocks":296,"cache_hit":0,"new_prefill":27639,"score_linear":14329.0,"score_lmetric":83334},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":71575,"ongoing_decode_tokens":15283,"pending_prefill_tokens":56292,"num_requests":2,"active_p_offloads":0,"cached_blocks":231,"cache_hit":0,"new_prefill":27639,"score_linear":71575.0,"score_lmetric":167862},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":258,"cache_hit":22528,"new_prefill":5111,"score_linear":-22528.0,"score_lmetric":-5111},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":125063,"ongoing_decode_tokens":0,"pending_prefill_tokens":117383,"num_requests":2,"active_p_offloads":0,"cached_blocks":36,"cache_hit":0,"new_prefill":27639,"score_linear":125063.0,"score_lmetric":290044},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":50783,"ongoing_decode_tokens":0,"pending_prefill_tokens":50783,"num_requests":1,"active_p_offloads":0,"cached_blocks":207,"cache_hit":0,"new_prefill":27639,"score_linear":50783.0,"score_lmetric":78422},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":45884,"ongoing_decode_tokens":0,"pending_prefill_tokens":23868,"num_requests":2,"active_p_offloads":0,"cached_blocks":141,"cache_hit":0,"new_prefill":27639,"score_linear":45884.0,"score_lmetric":103014}],"chosen_score_linear":-22528.0,"chosen_score_lmetric":-5111,"t_first_token":348406.058595907,"t_first_token_unix":1779779271.0725396,"t_done":348502.628683097,"t_done_unix":1779779367.6426265},{"request_id":"1290365:3:1293216:222","session_id":"1290365","input_length":24482,"t_proxy_recv":348500.898165143,"t_decision_unix":1779779365.9121068,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":23040,"affinity_cache_ratio":0.941099583367372,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":23040,"estimated_new_tokens":1442,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":512,"cache_hit":0,"new_prefill":24482,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":77805,"ongoing_decode_tokens":0,"pending_prefill_tokens":77805,"num_requests":1,"active_p_offloads":0,"cached_blocks":534,"cache_hit":0,"new_prefill":24482,"score_linear":77805.0,"score_lmetric":102287},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":631,"cache_hit":0,"new_prefill":24482,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":673,"cache_hit":0,"new_prefill":24482,"score_linear":82014.0,"score_lmetric":106496},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":743,"cache_hit":0,"new_prefill":24482,"score_linear":27639.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":499,"cache_hit":23040,"new_prefill":1442,"score_linear":-23040.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80130,"ongoing_decode_tokens":80130,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":691,"cache_hit":0,"new_prefill":24482,"score_linear":80130.0,"score_lmetric":48964},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":202906,"ongoing_decode_tokens":172644,"pending_prefill_tokens":12854,"num_requests":3,"active_p_offloads":0,"cached_blocks":426,"cache_hit":0,"new_prefill":24482,"score_linear":202906.0,"score_lmetric":112008}],"chosen_score_linear":-23040.0,"chosen_score_lmetric":0,"t_first_token":348501.161939555,"t_first_token_unix":1779779366.1758833,"t_done":348503.720470249,"t_done_unix":1779779368.7344139},{"request_id":"1291944:2:1293798:227","session_id":"1291944","input_length":12388,"t_proxy_recv":348503.029380984,"t_decision_unix":1779779368.0433226,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":7680,"affinity_cache_ratio":0.6199547949628673,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":7680,"estimated_new_tokens":4708,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":69074,"ongoing_decode_tokens":0,"pending_prefill_tokens":20946,"num_requests":2,"active_p_offloads":0,"cached_blocks":512,"cache_hit":0,"new_prefill":12388,"score_linear":69074.0,"score_lmetric":66668},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":686,"cache_hit":0,"new_prefill":12388,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":633,"cache_hit":0,"new_prefill":12388,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":673,"cache_hit":0,"new_prefill":12388,"score_linear":82014.0,"score_lmetric":94402},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":12388,"score_linear":0.0,"score_lmetric":-12388},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":24482,"ongoing_decode_tokens":24482,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":499,"cache_hit":7680,"new_prefill":4708,"score_linear":16802.0,"score_lmetric":4708},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":44920,"ongoing_decode_tokens":44920,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":691,"cache_hit":0,"new_prefill":12388,"score_linear":44920.0,"score_lmetric":12388},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":202906,"ongoing_decode_tokens":202906,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":426,"cache_hit":0,"new_prefill":12388,"score_linear":202906.0,"score_lmetric":37164}],"chosen_score_linear":16802.0,"chosen_score_lmetric":4708,"t_first_token":348503.505408922,"t_first_token_unix":1779779368.5193524,"t_done":348503.911717963,"t_done_unix":1779779368.925661},{"request_id":"1262354:14:1294243:228","session_id":"1262354","input_length":35435,"t_proxy_recv":348504.197978888,"t_decision_unix":1779779369.21192,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":34816,"affinity_cache_ratio":0.9825313955129109,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":34816,"estimated_new_tokens":619,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":69074,"ongoing_decode_tokens":0,"pending_prefill_tokens":20946,"num_requests":2,"active_p_offloads":0,"cached_blocks":512,"cache_hit":12800,"new_prefill":22635,"score_linear":56274.0,"score_lmetric":87162},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":686,"cache_hit":0,"new_prefill":35435,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":633,"cache_hit":0,"new_prefill":35435,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":673,"cache_hit":0,"new_prefill":35435,"score_linear":82014.0,"score_lmetric":117449},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":35435,"score_linear":0.0,"score_lmetric":-35435},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":511,"cache_hit":0,"new_prefill":35435,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":44920,"ongoing_decode_tokens":44920,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":691,"cache_hit":34816,"new_prefill":619,"score_linear":10104.0,"score_lmetric":619},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":202906,"ongoing_decode_tokens":202906,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":426,"cache_hit":28672,"new_prefill":6763,"score_linear":174234.0,"score_lmetric":20289}],"chosen_score_linear":10104.0,"chosen_score_lmetric":619,"t_first_token":348504.334111612,"t_first_token_unix":1779779369.3480551,"t_done":348506.367370679,"t_done_unix":1779779371.381314},{"request_id":"1269373:4:1280765:165","session_id":"1269373","input_length":30262,"t_proxy_recv":348499.185435016,"t_decision_unix":1779779364.199376,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":17408,"affinity_cache_ratio":0.5752428788579736,"affinity_num_requests":2,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":17408,"estimated_new_tokens":12854,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":15969,"ongoing_decode_tokens":15969,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":511,"cache_hit":7680,"new_prefill":22582,"score_linear":8289.0,"score_lmetric":22582},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":77805,"ongoing_decode_tokens":0,"pending_prefill_tokens":77805,"num_requests":1,"active_p_offloads":0,"cached_blocks":534,"cache_hit":0,"new_prefill":30262,"score_linear":77805.0,"score_lmetric":108067},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":16914,"ongoing_decode_tokens":0,"pending_prefill_tokens":7698,"num_requests":1,"active_p_offloads":0,"cached_blocks":623,"cache_hit":0,"new_prefill":30262,"score_linear":16914.0,"score_lmetric":37960},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":673,"cache_hit":0,"new_prefill":30262,"score_linear":82014.0,"score_lmetric":112276},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27639,"ongoing_decode_tokens":27639,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":734,"cache_hit":0,"new_prefill":30262,"score_linear":27639.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":499,"cache_hit":0,"new_prefill":30262,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":44920,"ongoing_decode_tokens":44920,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":691,"cache_hit":7680,"new_prefill":22582,"score_linear":37240.0,"score_lmetric":22582},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":172644,"ongoing_decode_tokens":172644,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":426,"cache_hit":17408,"new_prefill":12854,"score_linear":155236.0,"score_lmetric":25708}],"chosen_score_linear":155236.0,"chosen_score_lmetric":25708,"t_first_token":348501.538146051,"t_first_token_unix":1779779366.5520937,"t_done":348507.369304169,"t_done_unix":1779779372.3832474},{"request_id":"1294611:1:1294611:229","session_id":"1294611","input_length":21947,"t_proxy_recv":348505.490433146,"t_decision_unix":1779779370.5043743,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":-9147,"tie_break_used":false,"cache_hit":12800,"estimated_new_tokens":9147,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":69074,"ongoing_decode_tokens":40278,"pending_prefill_tokens":12924,"num_requests":2,"active_p_offloads":0,"cached_blocks":512,"cache_hit":0,"new_prefill":21947,"score_linear":69074.0,"score_lmetric":69742},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":686,"cache_hit":0,"new_prefill":21947,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":633,"cache_hit":0,"new_prefill":21947,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":673,"cache_hit":0,"new_prefill":21947,"score_linear":82014.0,"score_lmetric":103961},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":765,"cache_hit":12800,"new_prefill":9147,"score_linear":-12800.0,"score_lmetric":-9147},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":511,"cache_hit":0,"new_prefill":21947,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80355,"ongoing_decode_tokens":80355,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":691,"cache_hit":0,"new_prefill":21947,"score_linear":80355.0,"score_lmetric":43894},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":202906,"ongoing_decode_tokens":202906,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":426,"cache_hit":12800,"new_prefill":9147,"score_linear":190106.0,"score_lmetric":27441}],"chosen_score_linear":-12800.0,"chosen_score_lmetric":-9147,"t_first_token":348506.884238863,"t_first_token_unix":1779779371.8981824,"t_done":348507.555983292,"t_done_unix":1779779372.5699263},{"request_id":"1266668:5:1281133:168","session_id":"1266668","input_length":58032,"t_proxy_recv":348495.061824747,"t_decision_unix":1779779360.0757663,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":49664,"affinity_cache_ratio":0.8558036945133719,"affinity_num_requests":2,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":49664,"estimated_new_tokens":8368,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":14885,"ongoing_decode_tokens":14885,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":508,"cache_hit":7680,"new_prefill":50352,"score_linear":7205.0,"score_lmetric":50352},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":77805,"ongoing_decode_tokens":0,"pending_prefill_tokens":77805,"num_requests":1,"active_p_offloads":0,"cached_blocks":534,"cache_hit":0,"new_prefill":58032,"score_linear":77805.0,"score_lmetric":135837},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":623,"cache_hit":0,"new_prefill":58032,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":673,"cache_hit":0,"new_prefill":58032,"score_linear":82014.0,"score_lmetric":140046},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":116865,"ongoing_decode_tokens":116865,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":732,"cache_hit":0,"new_prefill":58032,"score_linear":116865.0,"score_lmetric":58032},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":484,"cache_hit":0,"new_prefill":58032,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":44920,"ongoing_decode_tokens":44920,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":691,"cache_hit":7680,"new_prefill":50352,"score_linear":37240.0,"score_lmetric":50352},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":131512,"ongoing_decode_tokens":131512,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":421,"cache_hit":49664,"new_prefill":8368,"score_linear":81848.0,"score_lmetric":16736}],"chosen_score_linear":81848.0,"chosen_score_lmetric":16736,"t_first_token":348497.981927169,"t_first_token_unix":1779779362.9958704,"t_done":348508.10191118,"t_done_unix":1779779373.1158547},{"request_id":"1272313:8:1293738:226","session_id":"1272313","input_length":28796,"t_proxy_recv":348502.882437327,"t_decision_unix":1779779367.896378,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":15872,"affinity_cache_ratio":0.5511876649534657,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":15872,"estimated_new_tokens":12924,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":40278,"ongoing_decode_tokens":0,"pending_prefill_tokens":8022,"num_requests":1,"active_p_offloads":0,"cached_blocks":512,"cache_hit":15872,"new_prefill":12924,"score_linear":24406.0,"score_lmetric":20946},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":686,"cache_hit":0,"new_prefill":28796,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":633,"cache_hit":0,"new_prefill":28796,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":673,"cache_hit":0,"new_prefill":28796,"score_linear":82014.0,"score_lmetric":110810},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":28796,"score_linear":0.0,"score_lmetric":-28796},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":24482,"ongoing_decode_tokens":24482,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":499,"cache_hit":0,"new_prefill":28796,"score_linear":24482.0,"score_lmetric":28796},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":44920,"ongoing_decode_tokens":44920,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":691,"cache_hit":7680,"new_prefill":21116,"score_linear":37240.0,"score_lmetric":21116},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":202906,"ongoing_decode_tokens":202906,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":426,"cache_hit":10752,"new_prefill":18044,"score_linear":192154.0,"score_lmetric":54132}],"chosen_score_linear":24406.0,"chosen_score_lmetric":20946,"t_first_token":348506.688733696,"t_first_token_unix":1779779371.7026772,"t_done":348509.25791988,"t_done_unix":1779779374.2718632},{"request_id":"1279412:6:1293655:225","session_id":"1279412","input_length":40278,"t_proxy_recv":348502.524063667,"t_decision_unix":1779779367.5380046,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":32256,"affinity_cache_ratio":0.8008342022940563,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":32256,"estimated_new_tokens":8022,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":512,"cache_hit":32256,"new_prefill":8022,"score_linear":-32256.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":686,"cache_hit":0,"new_prefill":40278,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":633,"cache_hit":0,"new_prefill":40278,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":673,"cache_hit":0,"new_prefill":40278,"score_linear":82014.0,"score_lmetric":122292},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":34099,"ongoing_decode_tokens":34099,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":743,"cache_hit":0,"new_prefill":40278,"score_linear":34099.0,"score_lmetric":40278},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":24482,"ongoing_decode_tokens":24482,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":499,"cache_hit":0,"new_prefill":40278,"score_linear":24482.0,"score_lmetric":40278},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":44920,"ongoing_decode_tokens":44920,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":691,"cache_hit":12800,"new_prefill":27478,"score_linear":32120.0,"score_lmetric":27478},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":202906,"ongoing_decode_tokens":202906,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":426,"cache_hit":12800,"new_prefill":27478,"score_linear":190106.0,"score_lmetric":82434}],"chosen_score_linear":-32256.0,"chosen_score_lmetric":0,"t_first_token":348504.544283843,"t_first_token_unix":1779779369.5582273,"t_done":348510.776525629,"t_done_unix":1779779375.790469},{"request_id":"1295690:1:1295690:232","session_id":"1295690","input_length":10857,"t_proxy_recv":348509.39406638,"t_decision_unix":1779779374.4080076,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":-10857,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":10857,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":40278,"ongoing_decode_tokens":40278,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":537,"cache_hit":0,"new_prefill":10857,"score_linear":40278.0,"score_lmetric":10857},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":686,"cache_hit":0,"new_prefill":10857,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":633,"cache_hit":0,"new_prefill":10857,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":673,"cache_hit":0,"new_prefill":10857,"score_linear":82014.0,"score_lmetric":92871},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":783,"cache_hit":0,"new_prefill":10857,"score_linear":0.0,"score_lmetric":-10857},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":511,"cache_hit":0,"new_prefill":10857,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":44920,"ongoing_decode_tokens":44920,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":692,"cache_hit":0,"new_prefill":10857,"score_linear":44920.0,"score_lmetric":10857},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":225261,"ongoing_decode_tokens":145206,"pending_prefill_tokens":21687,"num_requests":3,"active_p_offloads":0,"cached_blocks":468,"cache_hit":0,"new_prefill":10857,"score_linear":225261.0,"score_lmetric":97632}],"chosen_score_linear":0.0,"chosen_score_lmetric":-10857,"t_first_token":348510.335080258,"t_first_token_unix":1779779375.3490238,"t_done":348511.208331267,"t_done_unix":1779779376.2222745},{"request_id":"1289570:1:1289570:204","session_id":"1289570","input_length":82014,"t_proxy_recv":348488.74081212,"t_decision_unix":1779779353.7547536,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":82014,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":32191,"ongoing_decode_tokens":32191,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":488,"cache_hit":0,"new_prefill":82014,"score_linear":32191.0,"score_lmetric":82014},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":77805,"ongoing_decode_tokens":0,"pending_prefill_tokens":77805,"num_requests":1,"active_p_offloads":0,"cached_blocks":534,"cache_hit":0,"new_prefill":82014,"score_linear":77805.0,"score_lmetric":159819},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":9610,"ongoing_decode_tokens":0,"pending_prefill_tokens":8074,"num_requests":1,"active_p_offloads":0,"cached_blocks":596,"cache_hit":0,"new_prefill":82014,"score_linear":9610.0,"score_lmetric":90088},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":673,"cache_hit":0,"new_prefill":82014,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":158843,"ongoing_decode_tokens":158843,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":712,"cache_hit":0,"new_prefill":82014,"score_linear":158843.0,"score_lmetric":164028},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":443,"cache_hit":0,"new_prefill":82014,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":44920,"ongoing_decode_tokens":0,"pending_prefill_tokens":13688,"num_requests":1,"active_p_offloads":0,"cached_blocks":677,"cache_hit":0,"new_prefill":82014,"score_linear":44920.0,"score_lmetric":95702},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":191531,"ongoing_decode_tokens":191531,"pending_prefill_tokens":0,"num_requests":4,"active_p_offloads":0,"cached_blocks":388,"cache_hit":0,"new_prefill":82014,"score_linear":191531.0,"score_lmetric":328056}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348511.40032371,"t_first_token_unix":1779779376.4142673,"t_done":348511.924987147,"t_done_unix":1779779376.938931},{"request_id":"1294611:2:1295852:233","session_id":"1294611","input_length":23610,"t_proxy_recv":348509.898640342,"t_decision_unix":1779779374.9125817,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":22016,"affinity_cache_ratio":0.9324862346463363,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":22016,"estimated_new_tokens":1594,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":40278,"ongoing_decode_tokens":40278,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":537,"cache_hit":0,"new_prefill":23610,"score_linear":40278.0,"score_lmetric":23610},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":686,"cache_hit":0,"new_prefill":23610,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":633,"cache_hit":0,"new_prefill":23610,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":673,"cache_hit":0,"new_prefill":23610,"score_linear":82014.0,"score_lmetric":105624},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":10857,"ongoing_decode_tokens":0,"pending_prefill_tokens":10857,"num_requests":0,"active_p_offloads":0,"cached_blocks":783,"cache_hit":22016,"new_prefill":1594,"score_linear":-11159.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":511,"cache_hit":0,"new_prefill":23610,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":44920,"ongoing_decode_tokens":44920,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":692,"cache_hit":0,"new_prefill":23610,"score_linear":44920.0,"score_lmetric":23610},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":225261,"ongoing_decode_tokens":145206,"pending_prefill_tokens":21687,"num_requests":3,"active_p_offloads":0,"cached_blocks":468,"cache_hit":12800,"new_prefill":10810,"score_linear":212461.0,"score_lmetric":97491}],"chosen_score_linear":-11159.0,"chosen_score_lmetric":0,"t_first_token":348510.620256164,"t_first_token_unix":1779779375.6341996,"t_done":348512.092087319,"t_done_unix":1779779377.1060312},{"request_id":"1296872:1:1296872:235","session_id":"1296872","input_length":624,"t_proxy_recv":348513.457147587,"t_decision_unix":1779779378.4710886,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":-624,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":624,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":553,"cache_hit":0,"new_prefill":624,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":686,"cache_hit":0,"new_prefill":624,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":633,"cache_hit":0,"new_prefill":624,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":833,"cache_hit":0,"new_prefill":624,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":807,"cache_hit":0,"new_prefill":624,"score_linear":0.0,"score_lmetric":-624},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":511,"cache_hit":0,"new_prefill":624,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80589,"ongoing_decode_tokens":80589,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":692,"cache_hit":0,"new_prefill":624,"score_linear":80589.0,"score_lmetric":1248},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":225261,"ongoing_decode_tokens":145206,"pending_prefill_tokens":21687,"num_requests":3,"active_p_offloads":0,"cached_blocks":468,"cache_hit":0,"new_prefill":624,"score_linear":225261.0,"score_lmetric":66933}],"chosen_score_linear":0.0,"chosen_score_lmetric":-624,"t_first_token":348513.522228427,"t_first_token_unix":1779779378.536172,"t_done":348513.62569407,"t_done_unix":1779779378.639637},{"request_id":"1277909:3:1289205:201","session_id":"1277909","input_length":44920,"t_proxy_recv":348487.43562512,"t_decision_unix":1779779352.4495664,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":31232,"affinity_cache_ratio":0.6952804986642921,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":31232,"estimated_new_tokens":13688,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":488,"cache_hit":4096,"new_prefill":40824,"score_linear":-4096.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":77805,"ongoing_decode_tokens":0,"pending_prefill_tokens":77805,"num_requests":1,"active_p_offloads":0,"cached_blocks":534,"cache_hit":0,"new_prefill":44920,"score_linear":77805.0,"score_lmetric":122725},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":596,"cache_hit":0,"new_prefill":44920,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":673,"cache_hit":0,"new_prefill":44920,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":184143,"ongoing_decode_tokens":184143,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":537,"cache_hit":0,"new_prefill":44920,"score_linear":184143.0,"score_lmetric":134760},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":443,"cache_hit":0,"new_prefill":44920,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":677,"cache_hit":31232,"new_prefill":13688,"score_linear":-31232.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":191531,"ongoing_decode_tokens":191531,"pending_prefill_tokens":0,"num_requests":4,"active_p_offloads":0,"cached_blocks":388,"cache_hit":7680,"new_prefill":37240,"score_linear":183851.0,"score_lmetric":148960}],"chosen_score_linear":-31232.0,"chosen_score_lmetric":0,"t_first_token":348491.005863072,"t_first_token_unix":1779779356.0198078,"t_done":348513.918578067,"t_done_unix":1779779378.9325216},{"request_id":"1296983:1:1296983:236","session_id":"1296983","input_length":1791,"t_proxy_recv":348513.890683695,"t_decision_unix":1779779378.9046252,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":-1791,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":1791,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":553,"cache_hit":0,"new_prefill":1791,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":686,"cache_hit":0,"new_prefill":1791,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":633,"cache_hit":0,"new_prefill":1791,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":833,"cache_hit":0,"new_prefill":1791,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":808,"cache_hit":0,"new_prefill":1791,"score_linear":0.0,"score_lmetric":-1791},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":511,"cache_hit":0,"new_prefill":1791,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":80589,"ongoing_decode_tokens":80589,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":692,"cache_hit":0,"new_prefill":1791,"score_linear":80589.0,"score_lmetric":3582},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":225261,"ongoing_decode_tokens":145206,"pending_prefill_tokens":21687,"num_requests":3,"active_p_offloads":0,"cached_blocks":468,"cache_hit":0,"new_prefill":1791,"score_linear":225261.0,"score_lmetric":70434}],"chosen_score_linear":0.0,"chosen_score_lmetric":-1791,"t_first_token":348514.009780769,"t_first_token_unix":1779779379.0237243,"t_done":348514.236031545,"t_done_unix":1779779379.249975},{"request_id":"1262354:15:1296106:234","session_id":"1262354","input_length":35669,"t_proxy_recv":348510.846128184,"t_decision_unix":1779779375.8600695,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":35328,"affinity_cache_ratio":0.9904398777650061,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":35328,"estimated_new_tokens":341,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":553,"cache_hit":12800,"new_prefill":22869,"score_linear":-12800.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":686,"cache_hit":0,"new_prefill":35669,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":633,"cache_hit":0,"new_prefill":35669,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":673,"cache_hit":0,"new_prefill":35669,"score_linear":82014.0,"score_lmetric":117683},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":34467,"ongoing_decode_tokens":34467,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":783,"cache_hit":0,"new_prefill":35669,"score_linear":34467.0,"score_lmetric":35669},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":511,"cache_hit":0,"new_prefill":35669,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":44920,"ongoing_decode_tokens":44920,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":692,"cache_hit":35328,"new_prefill":341,"score_linear":9592.0,"score_lmetric":341},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":225261,"ongoing_decode_tokens":145206,"pending_prefill_tokens":21687,"num_requests":3,"active_p_offloads":0,"cached_blocks":468,"cache_hit":28672,"new_prefill":6997,"score_linear":196589.0,"score_lmetric":86052}],"chosen_score_linear":9592.0,"chosen_score_lmetric":341,"t_first_token":348510.933278383,"t_first_token_unix":1779779375.9472215,"t_done":348514.313020163,"t_done_unix":1779779379.3269637},{"request_id":"1260327:7:1297673:242","session_id":"1260327","input_length":42363,"t_proxy_recv":348516.080172206,"t_decision_unix":1779779381.094113,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":41984,"affinity_cache_ratio":0.99105351367939,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":41984,"estimated_new_tokens":379,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":60193,"ongoing_decode_tokens":0,"pending_prefill_tokens":19745,"num_requests":1,"active_p_offloads":0,"cached_blocks":553,"cache_hit":0,"new_prefill":42363,"score_linear":60193.0,"score_lmetric":62108},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":686,"cache_hit":0,"new_prefill":42363,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":62414,"ongoing_decode_tokens":0,"pending_prefill_tokens":43470,"num_requests":1,"active_p_offloads":0,"cached_blocks":633,"cache_hit":0,"new_prefill":42363,"score_linear":62414.0,"score_lmetric":85833},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":833,"cache_hit":0,"new_prefill":42363,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":83370,"ongoing_decode_tokens":0,"pending_prefill_tokens":59818,"num_requests":1,"active_p_offloads":0,"cached_blocks":811,"cache_hit":0,"new_prefill":42363,"score_linear":83370.0,"score_lmetric":102181},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":511,"cache_hit":0,"new_prefill":42363,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":723,"cache_hit":41984,"new_prefill":379,"score_linear":-41984.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":225261,"ongoing_decode_tokens":145206,"pending_prefill_tokens":21687,"num_requests":3,"active_p_offloads":0,"cached_blocks":468,"cache_hit":31232,"new_prefill":11131,"score_linear":194029.0,"score_lmetric":98454}],"chosen_score_linear":-41984.0,"chosen_score_lmetric":0,"t_first_token":348516.240950001,"t_first_token_unix":1779779381.2548933,"t_done":348516.470626683,"t_done_unix":1779779381.48457},{"request_id":"1290442:2:1298058:246","session_id":"1290442","input_length":11994,"t_proxy_recv":348517.427883791,"t_decision_unix":1779779382.4418254,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":7168,"affinity_cache_ratio":0.5976321494080373,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":7168,"estimated_new_tokens":4826,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":89342,"ongoing_decode_tokens":0,"pending_prefill_tokens":20222,"num_requests":2,"active_p_offloads":0,"cached_blocks":553,"cache_hit":0,"new_prefill":11994,"score_linear":89342.0,"score_lmetric":64432},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":21630,"ongoing_decode_tokens":0,"pending_prefill_tokens":8830,"num_requests":1,"active_p_offloads":0,"cached_blocks":686,"cache_hit":0,"new_prefill":11994,"score_linear":21630.0,"score_lmetric":20824},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":62414,"ongoing_decode_tokens":0,"pending_prefill_tokens":43470,"num_requests":1,"active_p_offloads":0,"cached_blocks":633,"cache_hit":0,"new_prefill":11994,"score_linear":62414.0,"score_lmetric":55464},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":833,"cache_hit":0,"new_prefill":11994,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":83370,"ongoing_decode_tokens":0,"pending_prefill_tokens":59818,"num_requests":1,"active_p_offloads":0,"cached_blocks":811,"cache_hit":0,"new_prefill":11994,"score_linear":83370.0,"score_lmetric":71812},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":13992,"ongoing_decode_tokens":0,"pending_prefill_tokens":12456,"num_requests":1,"active_p_offloads":0,"cached_blocks":511,"cache_hit":0,"new_prefill":11994,"score_linear":13992.0,"score_lmetric":24450},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":723,"cache_hit":7168,"new_prefill":4826,"score_linear":-7168.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":225261,"ongoing_decode_tokens":145206,"pending_prefill_tokens":21687,"num_requests":3,"active_p_offloads":0,"cached_blocks":468,"cache_hit":0,"new_prefill":11994,"score_linear":225261.0,"score_lmetric":101043}],"chosen_score_linear":-7168.0,"chosen_score_lmetric":0,"t_first_token":348517.929288602,"t_first_token_unix":1779779382.9432323,"t_done":348518.016949595,"t_done_unix":1779779383.0308936},{"request_id":"1297811:1:1297811:244","session_id":"1297811","input_length":13992,"t_proxy_recv":348516.556037999,"t_decision_unix":1779779381.5699797,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":1536,"estimated_new_tokens":12456,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":60193,"ongoing_decode_tokens":0,"pending_prefill_tokens":19745,"num_requests":1,"active_p_offloads":0,"cached_blocks":553,"cache_hit":0,"new_prefill":13992,"score_linear":60193.0,"score_lmetric":33737},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":686,"cache_hit":0,"new_prefill":13992,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":62414,"ongoing_decode_tokens":0,"pending_prefill_tokens":43470,"num_requests":1,"active_p_offloads":0,"cached_blocks":633,"cache_hit":0,"new_prefill":13992,"score_linear":62414.0,"score_lmetric":57462},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":833,"cache_hit":0,"new_prefill":13992,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":83370,"ongoing_decode_tokens":0,"pending_prefill_tokens":59818,"num_requests":1,"active_p_offloads":0,"cached_blocks":811,"cache_hit":0,"new_prefill":13992,"score_linear":83370.0,"score_lmetric":73810},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":511,"cache_hit":1536,"new_prefill":12456,"score_linear":-1536.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":723,"cache_hit":0,"new_prefill":13992,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":225261,"ongoing_decode_tokens":145206,"pending_prefill_tokens":21687,"num_requests":3,"active_p_offloads":0,"cached_blocks":468,"cache_hit":0,"new_prefill":13992,"score_linear":225261.0,"score_lmetric":107037}],"chosen_score_linear":-1536.0,"chosen_score_lmetric":0,"t_first_token":348517.700587034,"t_first_token_unix":1779779382.7145302,"t_done":348519.002549828,"t_done_unix":1779779384.0164933},{"request_id":"1290365:4:1298540:247","session_id":"1290365","input_length":25103,"t_proxy_recv":348519.134230549,"t_decision_unix":1779779384.1481717,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":24576,"affinity_cache_ratio":0.979006493247819,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":24576,"estimated_new_tokens":527,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":89342,"ongoing_decode_tokens":0,"pending_prefill_tokens":20222,"num_requests":2,"active_p_offloads":0,"cached_blocks":553,"cache_hit":0,"new_prefill":25103,"score_linear":89342.0,"score_lmetric":90650},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":21630,"ongoing_decode_tokens":21630,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":686,"cache_hit":0,"new_prefill":25103,"score_linear":21630.0,"score_lmetric":25103},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":62414,"ongoing_decode_tokens":0,"pending_prefill_tokens":43470,"num_requests":1,"active_p_offloads":0,"cached_blocks":633,"cache_hit":0,"new_prefill":25103,"score_linear":62414.0,"score_lmetric":68573},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":833,"cache_hit":0,"new_prefill":25103,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":83370,"ongoing_decode_tokens":0,"pending_prefill_tokens":59818,"num_requests":1,"active_p_offloads":0,"cached_blocks":811,"cache_hit":0,"new_prefill":25103,"score_linear":83370.0,"score_lmetric":84921},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":535,"cache_hit":24576,"new_prefill":527,"score_linear":-24576.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":732,"cache_hit":0,"new_prefill":25103,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":225261,"ongoing_decode_tokens":225261,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":468,"cache_hit":0,"new_prefill":25103,"score_linear":225261.0,"score_lmetric":75309}],"chosen_score_linear":-24576.0,"chosen_score_lmetric":0,"t_first_token":348519.242485042,"t_first_token_unix":1779779384.2564282,"t_done":348520.316677649,"t_done_unix":1779779385.3306215},{"request_id":"1268861:3:1298057:245","session_id":"1268861","input_length":21630,"t_proxy_recv":348517.416933086,"t_decision_unix":1779779382.4308746,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":12800,"affinity_cache_ratio":0.5917706888580675,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":12800,"estimated_new_tokens":8830,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":89342,"ongoing_decode_tokens":0,"pending_prefill_tokens":20222,"num_requests":2,"active_p_offloads":0,"cached_blocks":553,"cache_hit":0,"new_prefill":21630,"score_linear":89342.0,"score_lmetric":83704},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":686,"cache_hit":12800,"new_prefill":8830,"score_linear":-12800.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":62414,"ongoing_decode_tokens":0,"pending_prefill_tokens":43470,"num_requests":1,"active_p_offloads":0,"cached_blocks":633,"cache_hit":0,"new_prefill":21630,"score_linear":62414.0,"score_lmetric":65100},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":833,"cache_hit":0,"new_prefill":21630,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":83370,"ongoing_decode_tokens":0,"pending_prefill_tokens":59818,"num_requests":1,"active_p_offloads":0,"cached_blocks":811,"cache_hit":0,"new_prefill":21630,"score_linear":83370.0,"score_lmetric":81448},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":13992,"ongoing_decode_tokens":0,"pending_prefill_tokens":12456,"num_requests":1,"active_p_offloads":0,"cached_blocks":511,"cache_hit":0,"new_prefill":21630,"score_linear":13992.0,"score_lmetric":34086},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":723,"cache_hit":0,"new_prefill":21630,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":225261,"ongoing_decode_tokens":145206,"pending_prefill_tokens":21687,"num_requests":3,"active_p_offloads":0,"cached_blocks":468,"cache_hit":12288,"new_prefill":9342,"score_linear":212973.0,"score_lmetric":93087}],"chosen_score_linear":-12800.0,"chosen_score_lmetric":0,"t_first_token":348518.680967972,"t_first_token_unix":1779779383.6949117,"t_done":348520.85764481,"t_done_unix":1779779385.8715878},{"request_id":"1298970:1:1298970:248","session_id":"1298970","input_length":9530,"t_proxy_recv":348520.69627495,"t_decision_unix":1779779385.7102163,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":9530,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":89342,"ongoing_decode_tokens":0,"pending_prefill_tokens":20222,"num_requests":2,"active_p_offloads":0,"cached_blocks":553,"cache_hit":0,"new_prefill":9530,"score_linear":89342.0,"score_lmetric":59504},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":21630,"ongoing_decode_tokens":21630,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":686,"cache_hit":0,"new_prefill":9530,"score_linear":21630.0,"score_lmetric":9530},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":62414,"ongoing_decode_tokens":0,"pending_prefill_tokens":43470,"num_requests":1,"active_p_offloads":0,"cached_blocks":633,"cache_hit":0,"new_prefill":9530,"score_linear":62414.0,"score_lmetric":53000},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":833,"cache_hit":0,"new_prefill":9530,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":83370,"ongoing_decode_tokens":0,"pending_prefill_tokens":59818,"num_requests":1,"active_p_offloads":0,"cached_blocks":811,"cache_hit":0,"new_prefill":9530,"score_linear":83370.0,"score_lmetric":69348},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":536,"cache_hit":0,"new_prefill":9530,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":732,"cache_hit":0,"new_prefill":9530,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":225261,"ongoing_decode_tokens":225261,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":468,"cache_hit":0,"new_prefill":9530,"score_linear":225261.0,"score_lmetric":28590}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348521.467361156,"t_first_token_unix":1779779386.481305,"t_done":348521.653729695,"t_done_unix":1779779386.667673},{"request_id":"1299073:1:1299073:249","session_id":"1299073","input_length":881,"t_proxy_recv":348521.056188513,"t_decision_unix":1779779386.07013,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":881,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":89342,"ongoing_decode_tokens":0,"pending_prefill_tokens":20222,"num_requests":2,"active_p_offloads":0,"cached_blocks":553,"cache_hit":0,"new_prefill":881,"score_linear":89342.0,"score_lmetric":42206},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":703,"cache_hit":0,"new_prefill":881,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":62414,"ongoing_decode_tokens":0,"pending_prefill_tokens":43470,"num_requests":1,"active_p_offloads":0,"cached_blocks":633,"cache_hit":0,"new_prefill":881,"score_linear":62414.0,"score_lmetric":44351},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":9530,"ongoing_decode_tokens":0,"pending_prefill_tokens":9530,"num_requests":1,"active_p_offloads":0,"cached_blocks":833,"cache_hit":0,"new_prefill":881,"score_linear":9530.0,"score_lmetric":10411},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":83370,"ongoing_decode_tokens":0,"pending_prefill_tokens":59818,"num_requests":1,"active_p_offloads":0,"cached_blocks":811,"cache_hit":0,"new_prefill":881,"score_linear":83370.0,"score_lmetric":60699},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":536,"cache_hit":0,"new_prefill":881,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":732,"cache_hit":0,"new_prefill":881,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":225261,"ongoing_decode_tokens":225261,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":468,"cache_hit":0,"new_prefill":881,"score_linear":225261.0,"score_lmetric":2643}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348521.176444934,"t_first_token_unix":1779779386.190388,"t_done":348522.303424265,"t_done_unix":1779779387.3173673},{"request_id":"1296872:2:1299394:251","session_id":"1296872","input_length":8352,"t_proxy_recv":348522.014586844,"t_decision_unix":1779779387.0285282,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":4,"chosen_idx":3,"affinity_cache_hit":512,"affinity_cache_ratio":0.06130268199233716,"affinity_num_requests":1,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":8352,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":89342,"ongoing_decode_tokens":89342,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":553,"cache_hit":0,"new_prefill":8352,"score_linear":89342.0,"score_lmetric":16704},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":703,"cache_hit":0,"new_prefill":8352,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":62414,"ongoing_decode_tokens":0,"pending_prefill_tokens":43470,"num_requests":1,"active_p_offloads":0,"cached_blocks":633,"cache_hit":0,"new_prefill":8352,"score_linear":62414.0,"score_lmetric":51822},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":851,"cache_hit":0,"new_prefill":8352,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":83370,"ongoing_decode_tokens":0,"pending_prefill_tokens":59818,"num_requests":1,"active_p_offloads":0,"cached_blocks":811,"cache_hit":512,"new_prefill":7840,"score_linear":82858.0,"score_lmetric":67658},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":881,"ongoing_decode_tokens":881,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":536,"cache_hit":0,"new_prefill":8352,"score_linear":881.0,"score_lmetric":8352},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":7963,"ongoing_decode_tokens":0,"pending_prefill_tokens":7963,"num_requests":1,"active_p_offloads":0,"cached_blocks":732,"cache_hit":0,"new_prefill":8352,"score_linear":7963.0,"score_lmetric":16315},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":225261,"ongoing_decode_tokens":225261,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":468,"cache_hit":0,"new_prefill":8352,"score_linear":225261.0,"score_lmetric":25056}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348522.62169696,"t_first_token_unix":1779779387.635648,"t_done":348522.75268288,"t_done_unix":1779779387.7666266},{"request_id":"1272313:9:1297810:243","session_id":"1272313","input_length":29149,"t_proxy_recv":348516.559966526,"t_decision_unix":1779779381.5739083,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":28672,"affinity_cache_ratio":0.9836358022573674,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":28672,"estimated_new_tokens":477,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":60193,"ongoing_decode_tokens":0,"pending_prefill_tokens":19745,"num_requests":1,"active_p_offloads":0,"cached_blocks":553,"cache_hit":28672,"new_prefill":477,"score_linear":31521.0,"score_lmetric":20222},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":686,"cache_hit":0,"new_prefill":29149,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":62414,"ongoing_decode_tokens":0,"pending_prefill_tokens":43470,"num_requests":1,"active_p_offloads":0,"cached_blocks":633,"cache_hit":0,"new_prefill":29149,"score_linear":62414.0,"score_lmetric":72619},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":833,"cache_hit":0,"new_prefill":29149,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":83370,"ongoing_decode_tokens":0,"pending_prefill_tokens":59818,"num_requests":1,"active_p_offloads":0,"cached_blocks":811,"cache_hit":0,"new_prefill":29149,"score_linear":83370.0,"score_lmetric":88967},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":13992,"ongoing_decode_tokens":0,"pending_prefill_tokens":12456,"num_requests":1,"active_p_offloads":0,"cached_blocks":511,"cache_hit":0,"new_prefill":29149,"score_linear":13992.0,"score_lmetric":41605},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":723,"cache_hit":7680,"new_prefill":21469,"score_linear":-7680.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":225261,"ongoing_decode_tokens":145206,"pending_prefill_tokens":21687,"num_requests":3,"active_p_offloads":0,"cached_blocks":468,"cache_hit":10752,"new_prefill":18397,"score_linear":214509.0,"score_lmetric":120252}],"chosen_score_linear":31521.0,"chosen_score_lmetric":20222,"t_first_token":348521.431151121,"t_first_token_unix":1779779386.4450946,"t_done":348522.940476496,"t_done_unix":1779779387.9544196},{"request_id":"1279412:7:1297236:238","session_id":"1279412","input_length":60193,"t_proxy_recv":348514.895065385,"t_decision_unix":1779779379.9090066,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":40448,"affinity_cache_ratio":0.6719718239662419,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":40448,"estimated_new_tokens":19745,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":553,"cache_hit":40448,"new_prefill":19745,"score_linear":-40448.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":686,"cache_hit":0,"new_prefill":60193,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":633,"cache_hit":0,"new_prefill":60193,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":833,"cache_hit":0,"new_prefill":60193,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":59507,"ongoing_decode_tokens":0,"pending_prefill_tokens":59507,"num_requests":0,"active_p_offloads":0,"cached_blocks":811,"cache_hit":0,"new_prefill":60193,"score_linear":59507.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":511,"cache_hit":0,"new_prefill":60193,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":723,"cache_hit":12800,"new_prefill":47393,"score_linear":-12800.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":225261,"ongoing_decode_tokens":145206,"pending_prefill_tokens":21687,"num_requests":3,"active_p_offloads":0,"cached_blocks":468,"cache_hit":12800,"new_prefill":47393,"score_linear":212461.0,"score_lmetric":207240}],"chosen_score_linear":-40448.0,"chosen_score_lmetric":0,"t_first_token":348521.42970323,"t_first_token_unix":1779779386.443647,"t_done":348523.735697195,"t_done_unix":1779779388.7496405},{"request_id":"1292750:2:1299406:252","session_id":"1292750","input_length":19157,"t_proxy_recv":348522.03636653,"t_decision_unix":1779779387.0503075,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":4,"chosen_idx":1,"affinity_cache_hit":4608,"affinity_cache_ratio":0.24053870647804979,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":19157,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":89342,"ongoing_decode_tokens":89342,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":553,"cache_hit":0,"new_prefill":19157,"score_linear":89342.0,"score_lmetric":38314},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":703,"cache_hit":0,"new_prefill":19157,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":62414,"ongoing_decode_tokens":0,"pending_prefill_tokens":43470,"num_requests":1,"active_p_offloads":0,"cached_blocks":633,"cache_hit":0,"new_prefill":19157,"score_linear":62414.0,"score_lmetric":62627},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":8352,"ongoing_decode_tokens":0,"pending_prefill_tokens":8352,"num_requests":1,"active_p_offloads":0,"cached_blocks":851,"cache_hit":0,"new_prefill":19157,"score_linear":8352.0,"score_lmetric":27509},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":83370,"ongoing_decode_tokens":0,"pending_prefill_tokens":59818,"num_requests":1,"active_p_offloads":0,"cached_blocks":811,"cache_hit":4608,"new_prefill":14549,"score_linear":78762.0,"score_lmetric":74367},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":881,"ongoing_decode_tokens":881,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":536,"cache_hit":0,"new_prefill":19157,"score_linear":881.0,"score_lmetric":19157},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":7963,"ongoing_decode_tokens":0,"pending_prefill_tokens":7963,"num_requests":1,"active_p_offloads":0,"cached_blocks":732,"cache_hit":0,"new_prefill":19157,"score_linear":7963.0,"score_lmetric":27120},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":225261,"ongoing_decode_tokens":225261,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":468,"cache_hit":0,"new_prefill":19157,"score_linear":225261.0,"score_lmetric":57471}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348523.964184568,"t_first_token_unix":1779779388.978128,"t_done":348524.053855862,"t_done_unix":1779779389.067799},{"request_id":"1296983:2:1299368:250","session_id":"1296983","input_length":7963,"t_proxy_recv":348521.881214859,"t_decision_unix":1779779386.8951564,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":4,"chosen_idx":6,"affinity_cache_hit":1536,"affinity_cache_ratio":0.1928921260831345,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":7963,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":89342,"ongoing_decode_tokens":89342,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":553,"cache_hit":0,"new_prefill":7963,"score_linear":89342.0,"score_lmetric":15926},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":703,"cache_hit":0,"new_prefill":7963,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":62414,"ongoing_decode_tokens":0,"pending_prefill_tokens":43470,"num_requests":1,"active_p_offloads":0,"cached_blocks":633,"cache_hit":0,"new_prefill":7963,"score_linear":62414.0,"score_lmetric":51433},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":851,"cache_hit":0,"new_prefill":7963,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":83370,"ongoing_decode_tokens":0,"pending_prefill_tokens":59818,"num_requests":1,"active_p_offloads":0,"cached_blocks":811,"cache_hit":1536,"new_prefill":6427,"score_linear":81834.0,"score_lmetric":66245},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":881,"ongoing_decode_tokens":881,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":536,"cache_hit":0,"new_prefill":7963,"score_linear":881.0,"score_lmetric":7963},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":732,"cache_hit":0,"new_prefill":7963,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":225261,"ongoing_decode_tokens":225261,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":468,"cache_hit":0,"new_prefill":7963,"score_linear":225261.0,"score_lmetric":23889}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348522.443024747,"t_first_token_unix":1779779387.4569678,"t_done":348524.567112458,"t_done_unix":1779779389.5810566},{"request_id":"1286804:2:1300234:256","session_id":"1286804","input_length":14213,"t_proxy_recv":348524.861786569,"t_decision_unix":1779779389.875728,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":13824,"affinity_cache_ratio":0.9726306902131852,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":13824,"estimated_new_tokens":389,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":90164,"ongoing_decode_tokens":90164,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":593,"cache_hit":0,"new_prefill":14213,"score_linear":90164.0,"score_lmetric":28426},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":740,"cache_hit":0,"new_prefill":14213,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":62414,"ongoing_decode_tokens":0,"pending_prefill_tokens":43470,"num_requests":1,"active_p_offloads":0,"cached_blocks":633,"cache_hit":0,"new_prefill":14213,"score_linear":62414.0,"score_lmetric":57683},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":867,"cache_hit":0,"new_prefill":14213,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":83370,"ongoing_decode_tokens":0,"pending_prefill_tokens":59818,"num_requests":1,"active_p_offloads":0,"cached_blocks":811,"cache_hit":0,"new_prefill":14213,"score_linear":83370.0,"score_lmetric":74031},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":538,"cache_hit":13824,"new_prefill":389,"score_linear":-13824.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":18893,"ongoing_decode_tokens":18893,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":747,"cache_hit":0,"new_prefill":14213,"score_linear":18893.0,"score_lmetric":14213},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":225261,"ongoing_decode_tokens":225261,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":468,"cache_hit":0,"new_prefill":14213,"score_linear":225261.0,"score_lmetric":42639}],"chosen_score_linear":-13824.0,"chosen_score_lmetric":0,"t_first_token":348524.921683518,"t_first_token_unix":1779779389.9356265,"t_done":348525.568699583,"t_done_unix":1779779390.5826428},{"request_id":"1272313:10:1299814:254","session_id":"1272313","input_length":29486,"t_proxy_recv":348523.41493085,"t_decision_unix":1779779388.4288719,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":29184,"affinity_cache_ratio":0.9897578511836126,"affinity_num_requests":1,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":29184,"estimated_new_tokens":302,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":60193,"ongoing_decode_tokens":60193,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":554,"cache_hit":29184,"new_prefill":302,"score_linear":31009.0,"score_lmetric":302},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":19157,"ongoing_decode_tokens":0,"pending_prefill_tokens":19157,"num_requests":1,"active_p_offloads":0,"cached_blocks":703,"cache_hit":0,"new_prefill":29486,"score_linear":19157.0,"score_lmetric":48643},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":62414,"ongoing_decode_tokens":0,"pending_prefill_tokens":43470,"num_requests":1,"active_p_offloads":0,"cached_blocks":633,"cache_hit":0,"new_prefill":29486,"score_linear":62414.0,"score_lmetric":72956},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":867,"cache_hit":0,"new_prefill":29486,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":83370,"ongoing_decode_tokens":0,"pending_prefill_tokens":59818,"num_requests":1,"active_p_offloads":0,"cached_blocks":811,"cache_hit":0,"new_prefill":29486,"score_linear":83370.0,"score_lmetric":89304},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":538,"cache_hit":0,"new_prefill":29486,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":26856,"ongoing_decode_tokens":7963,"pending_prefill_tokens":7117,"num_requests":2,"active_p_offloads":0,"cached_blocks":732,"cache_hit":7680,"new_prefill":21806,"score_linear":19176.0,"score_lmetric":57846},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":225261,"ongoing_decode_tokens":225261,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":468,"cache_hit":10752,"new_prefill":18734,"score_linear":214509.0,"score_lmetric":56202}],"chosen_score_linear":31009.0,"chosen_score_lmetric":302,"t_first_token":348523.531810137,"t_first_token_unix":1779779388.545753,"t_done":348525.815407161,"t_done_unix":1779779390.8293502},{"request_id":"1279412:8:1300001:255","session_id":"1279412","input_length":60678,"t_proxy_recv":348524.100718821,"t_decision_unix":1779779389.11466,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":60416,"affinity_cache_ratio":0.9956821253172484,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":60416,"estimated_new_tokens":262,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":29486,"ongoing_decode_tokens":29486,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":593,"cache_hit":60416,"new_prefill":262,"score_linear":-30930.0,"score_lmetric":262},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":740,"cache_hit":0,"new_prefill":60678,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":62414,"ongoing_decode_tokens":0,"pending_prefill_tokens":43470,"num_requests":1,"active_p_offloads":0,"cached_blocks":633,"cache_hit":0,"new_prefill":60678,"score_linear":62414.0,"score_lmetric":104148},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":867,"cache_hit":0,"new_prefill":60678,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":83370,"ongoing_decode_tokens":0,"pending_prefill_tokens":59818,"num_requests":1,"active_p_offloads":0,"cached_blocks":811,"cache_hit":0,"new_prefill":60678,"score_linear":83370.0,"score_lmetric":120496},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":538,"cache_hit":0,"new_prefill":60678,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":26856,"ongoing_decode_tokens":7963,"pending_prefill_tokens":7117,"num_requests":2,"active_p_offloads":0,"cached_blocks":732,"cache_hit":12800,"new_prefill":47878,"score_linear":14056.0,"score_lmetric":109990},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":225261,"ongoing_decode_tokens":225261,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":468,"cache_hit":12800,"new_prefill":47878,"score_linear":212461.0,"score_lmetric":143634}],"chosen_score_linear":-30930.0,"chosen_score_lmetric":262,"t_first_token":348524.311923572,"t_first_token_unix":1779779389.3258674,"t_done":348526.396594145,"t_done_unix":1779779391.4105377},{"request_id":"1297473:1:1297473:239","session_id":"1297473","input_length":62414,"t_proxy_recv":348515.636741232,"t_decision_unix":1779779380.6506824,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":18944,"estimated_new_tokens":43470,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":60193,"ongoing_decode_tokens":0,"pending_prefill_tokens":19745,"num_requests":1,"active_p_offloads":0,"cached_blocks":553,"cache_hit":0,"new_prefill":62414,"score_linear":60193.0,"score_lmetric":82159},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":686,"cache_hit":0,"new_prefill":62414,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":633,"cache_hit":18944,"new_prefill":43470,"score_linear":-18944.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":833,"cache_hit":0,"new_prefill":62414,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":59507,"ongoing_decode_tokens":0,"pending_prefill_tokens":59507,"num_requests":0,"active_p_offloads":0,"cached_blocks":811,"cache_hit":0,"new_prefill":62414,"score_linear":59507.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":511,"cache_hit":0,"new_prefill":62414,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":723,"cache_hit":0,"new_prefill":62414,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":225261,"ongoing_decode_tokens":145206,"pending_prefill_tokens":21687,"num_requests":3,"active_p_offloads":0,"cached_blocks":468,"cache_hit":0,"new_prefill":62414,"score_linear":225261.0,"score_lmetric":252303}],"chosen_score_linear":-18944.0,"chosen_score_lmetric":0,"t_first_token":348527.729902328,"t_first_token_unix":1779779392.7438462,"t_done":348528.044785814,"t_done_unix":1779779393.0587292},{"request_id":"1269373:5:1282812:174","session_id":"1269373","input_length":30594,"t_proxy_recv":348507.376667788,"t_decision_unix":1779779372.3906062,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":30208,"affinity_cache_ratio":0.9873831470222919,"affinity_num_requests":2,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":30208,"estimated_new_tokens":386,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":69074,"ongoing_decode_tokens":69074,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":512,"cache_hit":7680,"new_prefill":22914,"score_linear":61394.0,"score_lmetric":45828},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":686,"cache_hit":0,"new_prefill":30594,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":633,"cache_hit":0,"new_prefill":30594,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":673,"cache_hit":0,"new_prefill":30594,"score_linear":82014.0,"score_lmetric":112608},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":21947,"ongoing_decode_tokens":21947,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":30594,"score_linear":21947.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":511,"cache_hit":0,"new_prefill":30594,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":44920,"ongoing_decode_tokens":44920,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":692,"cache_hit":7680,"new_prefill":22914,"score_linear":37240.0,"score_lmetric":22914},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":172644,"ongoing_decode_tokens":172644,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":451,"cache_hit":30208,"new_prefill":386,"score_linear":142436.0,"score_lmetric":772}],"chosen_score_linear":142436.0,"chosen_score_lmetric":772,"t_first_token":348507.501595345,"t_first_token_unix":1779779372.5155387,"t_done":348528.55364486,"t_done_unix":1779779393.5675883},{"request_id":"1297117:1:1297117:237","session_id":"1297117","input_length":59507,"t_proxy_recv":348514.37197016,"t_decision_unix":1779779379.3859115,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":-59507,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":59507,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":553,"cache_hit":0,"new_prefill":59507,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":686,"cache_hit":0,"new_prefill":59507,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":633,"cache_hit":0,"new_prefill":59507,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":833,"cache_hit":0,"new_prefill":59507,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":811,"cache_hit":0,"new_prefill":59507,"score_linear":0.0,"score_lmetric":-59507},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":511,"cache_hit":0,"new_prefill":59507,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":723,"cache_hit":0,"new_prefill":59507,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":225261,"ongoing_decode_tokens":145206,"pending_prefill_tokens":21687,"num_requests":3,"active_p_offloads":0,"cached_blocks":468,"cache_hit":0,"new_prefill":59507,"score_linear":225261.0,"score_lmetric":243582}],"chosen_score_linear":0.0,"chosen_score_lmetric":-59507,"t_first_token":348527.185524793,"t_first_token_unix":1779779392.1994681,"t_done":348529.41436049,"t_done_unix":1779779394.428304},{"request_id":"1299507:1:1299507:253","session_id":"1299507","input_length":18893,"t_proxy_recv":348522.296623204,"t_decision_unix":1779779387.3105645,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.375,"fallback_score":15080,"tie_break_used":false,"cache_hit":11776,"estimated_new_tokens":7117,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":89342,"ongoing_decode_tokens":89342,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":553,"cache_hit":0,"new_prefill":18893,"score_linear":89342.0,"score_lmetric":37786},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":19157,"ongoing_decode_tokens":0,"pending_prefill_tokens":19157,"num_requests":1,"active_p_offloads":0,"cached_blocks":703,"cache_hit":0,"new_prefill":18893,"score_linear":19157.0,"score_lmetric":38050},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":62414,"ongoing_decode_tokens":0,"pending_prefill_tokens":43470,"num_requests":1,"active_p_offloads":0,"cached_blocks":633,"cache_hit":0,"new_prefill":18893,"score_linear":62414.0,"score_lmetric":62363},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":8352,"ongoing_decode_tokens":0,"pending_prefill_tokens":8352,"num_requests":1,"active_p_offloads":0,"cached_blocks":851,"cache_hit":0,"new_prefill":18893,"score_linear":8352.0,"score_lmetric":27245},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":83370,"ongoing_decode_tokens":0,"pending_prefill_tokens":59818,"num_requests":1,"active_p_offloads":0,"cached_blocks":811,"cache_hit":0,"new_prefill":18893,"score_linear":83370.0,"score_lmetric":78711},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":881,"ongoing_decode_tokens":881,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":536,"cache_hit":0,"new_prefill":18893,"score_linear":881.0,"score_lmetric":18893},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":7963,"ongoing_decode_tokens":0,"pending_prefill_tokens":7963,"num_requests":1,"active_p_offloads":0,"cached_blocks":732,"cache_hit":11776,"new_prefill":7117,"score_linear":-3813.0,"score_lmetric":15080},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":225261,"ongoing_decode_tokens":225261,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":468,"cache_hit":0,"new_prefill":18893,"score_linear":225261.0,"score_lmetric":56679}],"chosen_score_linear":-3813.0,"chosen_score_lmetric":15080,"t_first_token":348524.3474109,"t_first_token_unix":1779779389.3613548,"t_done":348530.340871464,"t_done_unix":1779779395.3548148},{"request_id":"1266668:6:1284003:181","session_id":"1266668","input_length":80055,"t_proxy_recv":348508.123409945,"t_decision_unix":1779779373.1373518,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":58368,"affinity_cache_ratio":0.7290987446130786,"affinity_num_requests":2,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":58368,"estimated_new_tokens":21687,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":69074,"ongoing_decode_tokens":69074,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":512,"cache_hit":7680,"new_prefill":72375,"score_linear":61394.0,"score_lmetric":144750},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":686,"cache_hit":0,"new_prefill":80055,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":633,"cache_hit":0,"new_prefill":80055,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":82014,"ongoing_decode_tokens":0,"pending_prefill_tokens":82014,"num_requests":1,"active_p_offloads":0,"cached_blocks":673,"cache_hit":0,"new_prefill":80055,"score_linear":82014.0,"score_lmetric":162069},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":783,"cache_hit":0,"new_prefill":80055,"score_linear":0.0,"score_lmetric":-80055},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":511,"cache_hit":0,"new_prefill":80055,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":44920,"ongoing_decode_tokens":44920,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":692,"cache_hit":7680,"new_prefill":72375,"score_linear":37240.0,"score_lmetric":72375},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":145206,"ongoing_decode_tokens":145206,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":468,"cache_hit":58368,"new_prefill":21687,"score_linear":86838.0,"score_lmetric":43374}],"chosen_score_linear":86838.0,"chosen_score_lmetric":43374,"t_first_token":348517.630737552,"t_first_token_unix":1779779382.644684,"t_done":348530.596762968,"t_done_unix":1779779395.6107063},{"request_id":"1302071:1:1302071:267","session_id":"1302071","input_length":6527,"t_proxy_recv":348531.416125151,"t_decision_unix":1779779396.4300666,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":6527,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":34706,"ongoing_decode_tokens":0,"pending_prefill_tokens":31122,"num_requests":1,"active_p_offloads":0,"cached_blocks":594,"cache_hit":0,"new_prefill":6527,"score_linear":34706.0,"score_lmetric":37649},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":740,"cache_hit":0,"new_prefill":6527,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":26441,"ongoing_decode_tokens":0,"pending_prefill_tokens":26441,"num_requests":1,"active_p_offloads":0,"cached_blocks":717,"cache_hit":0,"new_prefill":6527,"score_linear":26441.0,"score_lmetric":32968},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":867,"cache_hit":0,"new_prefill":6527,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":52835,"ongoing_decode_tokens":23863,"pending_prefill_tokens":18220,"num_requests":1,"active_p_offloads":0,"cached_blocks":927,"cache_hit":0,"new_prefill":6527,"score_linear":52835.0,"score_lmetric":24747},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":69555,"ongoing_decode_tokens":0,"pending_prefill_tokens":30643,"num_requests":2,"active_p_offloads":0,"cached_blocks":538,"cache_hit":0,"new_prefill":6527,"score_linear":69555.0,"score_lmetric":74340},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":43181,"ongoing_decode_tokens":43181,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":762,"cache_hit":0,"new_prefill":6527,"score_linear":43181.0,"score_lmetric":6527},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":241141,"ongoing_decode_tokens":153400,"pending_prefill_tokens":7357,"num_requests":3,"active_p_offloads":0,"cached_blocks":512,"cache_hit":0,"new_prefill":6527,"score_linear":241141.0,"score_lmetric":41652}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348531.852944793,"t_first_token_unix":1779779396.8668883,"t_done":348532.094885176,"t_done_unix":1779779397.1088288},{"request_id":"1295690:2:1301706:264","session_id":"1295690","input_length":28972,"t_proxy_recv":348529.989378334,"t_decision_unix":1779779395.00332,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":10752,"affinity_cache_ratio":0.37111694049427035,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":10752,"estimated_new_tokens":18220,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":594,"cache_hit":0,"new_prefill":28972,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":740,"cache_hit":0,"new_prefill":28972,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":717,"cache_hit":0,"new_prefill":28972,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":867,"cache_hit":0,"new_prefill":28972,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":23863,"ongoing_decode_tokens":23863,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":927,"cache_hit":10752,"new_prefill":18220,"score_linear":13111.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":69555,"ongoing_decode_tokens":0,"pending_prefill_tokens":30643,"num_requests":2,"active_p_offloads":0,"cached_blocks":538,"cache_hit":0,"new_prefill":28972,"score_linear":69555.0,"score_lmetric":119230},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":62074,"ongoing_decode_tokens":18893,"pending_prefill_tokens":1197,"num_requests":2,"active_p_offloads":0,"cached_blocks":747,"cache_hit":0,"new_prefill":28972,"score_linear":62074.0,"score_lmetric":60338},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":233455,"ongoing_decode_tokens":194667,"pending_prefill_tokens":8068,"num_requests":3,"active_p_offloads":0,"cached_blocks":469,"cache_hit":0,"new_prefill":28972,"score_linear":233455.0,"score_lmetric":111120}],"chosen_score_linear":13111.0,"chosen_score_lmetric":0,"t_first_token":348532.92112705,"t_first_token_unix":1779779397.9350708,"t_done":348533.284688477,"t_done_unix":1779779398.298632},{"request_id":"1302449:1:1302449:269","session_id":"1302449","input_length":1105,"t_proxy_recv":348532.956941737,"t_decision_unix":1779779397.9708831,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":1105,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":34706,"ongoing_decode_tokens":0,"pending_prefill_tokens":31122,"num_requests":1,"active_p_offloads":0,"cached_blocks":594,"cache_hit":0,"new_prefill":1105,"score_linear":34706.0,"score_lmetric":32227},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":740,"cache_hit":0,"new_prefill":1105,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":26441,"ongoing_decode_tokens":0,"pending_prefill_tokens":26441,"num_requests":1,"active_p_offloads":0,"cached_blocks":717,"cache_hit":0,"new_prefill":1105,"score_linear":26441.0,"score_lmetric":27546},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":49855,"ongoing_decode_tokens":0,"pending_prefill_tokens":191,"num_requests":1,"active_p_offloads":0,"cached_blocks":879,"cache_hit":0,"new_prefill":1105,"score_linear":49855.0,"score_lmetric":1296},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":52835,"ongoing_decode_tokens":52835,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":927,"cache_hit":0,"new_prefill":1105,"score_linear":52835.0,"score_lmetric":1105},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":69555,"ongoing_decode_tokens":69555,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":538,"cache_hit":0,"new_prefill":1105,"score_linear":69555.0,"score_lmetric":2210},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":43181,"ongoing_decode_tokens":43181,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":762,"cache_hit":0,"new_prefill":1105,"score_linear":43181.0,"score_lmetric":1105},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":241141,"ongoing_decode_tokens":153400,"pending_prefill_tokens":7357,"num_requests":3,"active_p_offloads":0,"cached_blocks":512,"cache_hit":0,"new_prefill":1105,"score_linear":241141.0,"score_lmetric":25386}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348533.039328355,"t_first_token_unix":1779779398.0532718,"t_done":348533.484502216,"t_done_unix":1779779398.4984453},{"request_id":"1294611:3:1297534:240","session_id":"1294611","input_length":23863,"t_proxy_recv":348515.738335379,"t_decision_unix":1779779380.752277,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":23552,"affinity_cache_ratio":0.9869672715081926,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":23552,"estimated_new_tokens":311,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":60193,"ongoing_decode_tokens":0,"pending_prefill_tokens":19745,"num_requests":1,"active_p_offloads":0,"cached_blocks":553,"cache_hit":0,"new_prefill":23863,"score_linear":60193.0,"score_lmetric":43608},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":686,"cache_hit":0,"new_prefill":23863,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":62414,"ongoing_decode_tokens":0,"pending_prefill_tokens":43470,"num_requests":1,"active_p_offloads":0,"cached_blocks":633,"cache_hit":0,"new_prefill":23863,"score_linear":62414.0,"score_lmetric":67333},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":833,"cache_hit":0,"new_prefill":23863,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":59507,"ongoing_decode_tokens":0,"pending_prefill_tokens":59507,"num_requests":0,"active_p_offloads":0,"cached_blocks":811,"cache_hit":23552,"new_prefill":311,"score_linear":35955.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":511,"cache_hit":0,"new_prefill":23863,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":723,"cache_hit":0,"new_prefill":23863,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":225261,"ongoing_decode_tokens":145206,"pending_prefill_tokens":21687,"num_requests":3,"active_p_offloads":0,"cached_blocks":468,"cache_hit":12800,"new_prefill":11063,"score_linear":212461.0,"score_lmetric":98250}],"chosen_score_linear":35955.0,"chosen_score_lmetric":0,"t_first_token":348527.186640179,"t_first_token_unix":1779779392.2005847,"t_done":348534.067907962,"t_done_unix":1779779399.0818536},{"request_id":"1294611:4:1300505:257","session_id":"1294611","input_length":24384,"t_proxy_recv":348534.073776003,"t_decision_unix":1779779399.0877175,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":24064,"affinity_cache_ratio":0.9868766404199475,"affinity_num_requests":-1,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":24064,"estimated_new_tokens":320,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":64654,"ongoing_decode_tokens":0,"pending_prefill_tokens":31374,"num_requests":2,"active_p_offloads":0,"cached_blocks":594,"cache_hit":0,"new_prefill":24384,"score_linear":64654.0,"score_lmetric":111516},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":742,"cache_hit":0,"new_prefill":24384,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":26441,"ongoing_decode_tokens":0,"pending_prefill_tokens":26441,"num_requests":1,"active_p_offloads":0,"cached_blocks":717,"cache_hit":0,"new_prefill":24384,"score_linear":26441.0,"score_lmetric":50825},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":49855,"ongoing_decode_tokens":0,"pending_prefill_tokens":191,"num_requests":1,"active_p_offloads":0,"cached_blocks":879,"cache_hit":0,"new_prefill":24384,"score_linear":49855.0,"score_lmetric":24575},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":24064,"new_prefill":320,"score_linear":-24064.0,"score_lmetric":-320},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":69555,"ongoing_decode_tokens":69555,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":538,"cache_hit":0,"new_prefill":24384,"score_linear":69555.0,"score_lmetric":48768},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":43181,"ongoing_decode_tokens":43181,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":762,"cache_hit":0,"new_prefill":24384,"score_linear":43181.0,"score_lmetric":24384},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":241141,"ongoing_decode_tokens":153400,"pending_prefill_tokens":7357,"num_requests":3,"active_p_offloads":0,"cached_blocks":512,"cache_hit":12800,"new_prefill":11584,"score_linear":228341.0,"score_lmetric":56823}],"chosen_score_linear":-24064.0,"chosen_score_lmetric":-320,"t_first_token":348534.132319268,"t_first_token_unix":1779779399.1462626,"t_done":348534.496218454,"t_done_unix":1779779399.5101614},{"request_id":"1302063:1:1302063:266","session_id":"1302063","input_length":26441,"t_proxy_recv":348531.408287446,"t_decision_unix":1779779396.4222288,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":26441,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":34706,"ongoing_decode_tokens":0,"pending_prefill_tokens":31122,"num_requests":1,"active_p_offloads":0,"cached_blocks":594,"cache_hit":0,"new_prefill":26441,"score_linear":34706.0,"score_lmetric":57563},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":740,"cache_hit":0,"new_prefill":26441,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":717,"cache_hit":0,"new_prefill":26441,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":867,"cache_hit":0,"new_prefill":26441,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":52835,"ongoing_decode_tokens":23863,"pending_prefill_tokens":18220,"num_requests":1,"active_p_offloads":0,"cached_blocks":927,"cache_hit":0,"new_prefill":26441,"score_linear":52835.0,"score_lmetric":44661},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":69555,"ongoing_decode_tokens":0,"pending_prefill_tokens":30643,"num_requests":2,"active_p_offloads":0,"cached_blocks":538,"cache_hit":0,"new_prefill":26441,"score_linear":69555.0,"score_lmetric":114168},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":43181,"ongoing_decode_tokens":43181,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":762,"cache_hit":0,"new_prefill":26441,"score_linear":43181.0,"score_lmetric":26441},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":241141,"ongoing_decode_tokens":153400,"pending_prefill_tokens":7357,"num_requests":3,"active_p_offloads":0,"cached_blocks":512,"cache_hit":0,"new_prefill":26441,"score_linear":241141.0,"score_lmetric":101394}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348534.631754773,"t_first_token_unix":1779779399.645698,"t_done":348534.887125953,"t_done_unix":1779779399.9010694},{"request_id":"1294611:5:1301356:262","session_id":"1294611","input_length":25650,"t_proxy_recv":348534.504421326,"t_decision_unix":1779779399.5183625,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":24064,"affinity_cache_ratio":0.9381676413255361,"affinity_num_requests":-1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":24064,"estimated_new_tokens":1586,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":64654,"ongoing_decode_tokens":0,"pending_prefill_tokens":31374,"num_requests":2,"active_p_offloads":0,"cached_blocks":594,"cache_hit":0,"new_prefill":25650,"score_linear":64654.0,"score_lmetric":114048},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":15958,"ongoing_decode_tokens":0,"pending_prefill_tokens":15958,"num_requests":1,"active_p_offloads":0,"cached_blocks":742,"cache_hit":0,"new_prefill":25650,"score_linear":15958.0,"score_lmetric":41608},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":26441,"ongoing_decode_tokens":0,"pending_prefill_tokens":26441,"num_requests":1,"active_p_offloads":0,"cached_blocks":717,"cache_hit":0,"new_prefill":25650,"score_linear":26441.0,"score_lmetric":52091},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":49855,"ongoing_decode_tokens":0,"pending_prefill_tokens":191,"num_requests":1,"active_p_offloads":0,"cached_blocks":879,"cache_hit":0,"new_prefill":25650,"score_linear":49855.0,"score_lmetric":25841},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":24064,"new_prefill":1586,"score_linear":-24064.0,"score_lmetric":-1586},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":69555,"ongoing_decode_tokens":69555,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":538,"cache_hit":0,"new_prefill":25650,"score_linear":69555.0,"score_lmetric":51300},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":43181,"ongoing_decode_tokens":43181,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":762,"cache_hit":0,"new_prefill":25650,"score_linear":43181.0,"score_lmetric":25650},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":241141,"ongoing_decode_tokens":241141,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":512,"cache_hit":12800,"new_prefill":12850,"score_linear":228341.0,"score_lmetric":38550}],"chosen_score_linear":-24064.0,"chosen_score_lmetric":-1586,"t_first_token":348534.791048568,"t_first_token_unix":1779779399.8049917,"t_done":348535.099424601,"t_done_unix":1779779400.1133678},{"request_id":"1260327:8:1301686:263","session_id":"1260327","input_length":43181,"t_proxy_recv":348529.903474274,"t_decision_unix":1779779394.9174156,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":41984,"affinity_cache_ratio":0.9722794747689957,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":41984,"estimated_new_tokens":1197,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":594,"cache_hit":0,"new_prefill":43181,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":740,"cache_hit":0,"new_prefill":43181,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":717,"cache_hit":0,"new_prefill":43181,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":867,"cache_hit":0,"new_prefill":43181,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":23863,"ongoing_decode_tokens":23863,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":927,"cache_hit":0,"new_prefill":43181,"score_linear":23863.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":69555,"ongoing_decode_tokens":0,"pending_prefill_tokens":30643,"num_requests":2,"active_p_offloads":0,"cached_blocks":538,"cache_hit":0,"new_prefill":43181,"score_linear":69555.0,"score_lmetric":147648},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":18893,"ongoing_decode_tokens":18893,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":747,"cache_hit":41984,"new_prefill":1197,"score_linear":-23091.0,"score_lmetric":1197},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":233455,"ongoing_decode_tokens":194667,"pending_prefill_tokens":8068,"num_requests":3,"active_p_offloads":0,"cached_blocks":469,"cache_hit":31232,"new_prefill":11949,"score_linear":202223.0,"score_lmetric":60051}],"chosen_score_linear":-23091.0,"chosen_score_lmetric":1197,"t_first_token":348530.242166227,"t_first_token_unix":1779779395.2561095,"t_done":348535.131322088,"t_done_unix":1779779400.1452653},{"request_id":"1290365:5:1300807:259","session_id":"1290365","input_length":25948,"t_proxy_recv":348527.058415802,"t_decision_unix":1779779392.0723572,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":25088,"affinity_cache_ratio":0.9668567905040851,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":25088,"estimated_new_tokens":860,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":594,"cache_hit":0,"new_prefill":25948,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":740,"cache_hit":0,"new_prefill":25948,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":62414,"ongoing_decode_tokens":0,"pending_prefill_tokens":43470,"num_requests":1,"active_p_offloads":0,"cached_blocks":633,"cache_hit":0,"new_prefill":25948,"score_linear":62414.0,"score_lmetric":69418},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":867,"cache_hit":0,"new_prefill":25948,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":83370,"ongoing_decode_tokens":0,"pending_prefill_tokens":59818,"num_requests":1,"active_p_offloads":0,"cached_blocks":811,"cache_hit":0,"new_prefill":25948,"score_linear":83370.0,"score_lmetric":85766},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":43607,"ongoing_decode_tokens":0,"pending_prefill_tokens":29783,"num_requests":1,"active_p_offloads":0,"cached_blocks":538,"cache_hit":25088,"new_prefill":860,"score_linear":18519.0,"score_lmetric":30643},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":18893,"ongoing_decode_tokens":18893,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":747,"cache_hit":0,"new_prefill":25948,"score_linear":18893.0,"score_lmetric":25948},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":225261,"ongoing_decode_tokens":225261,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":468,"cache_hit":0,"new_prefill":25948,"score_linear":225261.0,"score_lmetric":77844}],"chosen_score_linear":18519.0,"chosen_score_lmetric":30643,"t_first_token":348532.485410372,"t_first_token_unix":1779779397.499354,"t_done":348535.551474369,"t_done_unix":1779779400.5654175},{"request_id":"1302907:1:1302907:272","session_id":"1302907","input_length":15958,"t_proxy_recv":348534.330775107,"t_decision_unix":1779779399.3447165,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":15958,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":64654,"ongoing_decode_tokens":0,"pending_prefill_tokens":31374,"num_requests":2,"active_p_offloads":0,"cached_blocks":594,"cache_hit":0,"new_prefill":15958,"score_linear":64654.0,"score_lmetric":94664},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":742,"cache_hit":0,"new_prefill":15958,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":26441,"ongoing_decode_tokens":0,"pending_prefill_tokens":26441,"num_requests":1,"active_p_offloads":0,"cached_blocks":717,"cache_hit":0,"new_prefill":15958,"score_linear":26441.0,"score_lmetric":42399},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":49855,"ongoing_decode_tokens":0,"pending_prefill_tokens":191,"num_requests":1,"active_p_offloads":0,"cached_blocks":879,"cache_hit":0,"new_prefill":15958,"score_linear":49855.0,"score_lmetric":16149},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":24384,"ongoing_decode_tokens":24384,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":963,"cache_hit":0,"new_prefill":15958,"score_linear":24384.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":69555,"ongoing_decode_tokens":69555,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":538,"cache_hit":0,"new_prefill":15958,"score_linear":69555.0,"score_lmetric":31916},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":43181,"ongoing_decode_tokens":43181,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":762,"cache_hit":0,"new_prefill":15958,"score_linear":43181.0,"score_lmetric":15958},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":241141,"ongoing_decode_tokens":153400,"pending_prefill_tokens":7357,"num_requests":3,"active_p_offloads":0,"cached_blocks":512,"cache_hit":0,"new_prefill":15958,"score_linear":241141.0,"score_lmetric":69945}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348535.788651304,"t_first_token_unix":1779779400.8025947,"t_done":348535.991552778,"t_done_unix":1779779401.005496},{"request_id":"1302449:2:1303563:277","session_id":"1302449","input_length":3709,"t_proxy_recv":348536.648225128,"t_decision_unix":1779779401.6621664,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":1024,"affinity_cache_ratio":0.27608519816662175,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":false,"cache_hit":1024,"estimated_new_tokens":2685,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":64654,"ongoing_decode_tokens":64654,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":594,"cache_hit":0,"new_prefill":3709,"score_linear":64654.0,"score_lmetric":7418},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":773,"cache_hit":1024,"new_prefill":2685,"score_linear":-1024.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":768,"cache_hit":0,"new_prefill":3709,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":49855,"ongoing_decode_tokens":0,"pending_prefill_tokens":191,"num_requests":1,"active_p_offloads":0,"cached_blocks":879,"cache_hit":0,"new_prefill":3709,"score_linear":49855.0,"score_lmetric":3900},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":45726,"ongoing_decode_tokens":30645,"pending_prefill_tokens":15081,"num_requests":1,"active_p_offloads":0,"cached_blocks":966,"cache_hit":0,"new_prefill":3709,"score_linear":45726.0,"score_lmetric":18790},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":540,"cache_hit":0,"new_prefill":3709,"score_linear":43607.0,"score_lmetric":3709},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":14944,"ongoing_decode_tokens":0,"pending_prefill_tokens":14944,"num_requests":1,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":3709,"score_linear":14944.0,"score_lmetric":18653},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":241141,"ongoing_decode_tokens":241141,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":512,"cache_hit":0,"new_prefill":3709,"score_linear":241141.0,"score_lmetric":11127}],"chosen_score_linear":-1024.0,"chosen_score_lmetric":0,"t_first_token":348536.827056101,"t_first_token_unix":1779779401.8409994,"t_done":348537.050623623,"t_done_unix":1779779402.0645673},{"request_id":"1303651:1:1303651:278","session_id":"1303651","input_length":3065,"t_proxy_recv":348536.959137731,"t_decision_unix":1779779401.9730792,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":3065,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":64654,"ongoing_decode_tokens":64654,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":594,"cache_hit":0,"new_prefill":3065,"score_linear":64654.0,"score_lmetric":6130},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":3709,"ongoing_decode_tokens":3709,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":773,"cache_hit":0,"new_prefill":3065,"score_linear":3709.0,"score_lmetric":3065},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":768,"cache_hit":0,"new_prefill":3065,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":49855,"ongoing_decode_tokens":0,"pending_prefill_tokens":191,"num_requests":1,"active_p_offloads":0,"cached_blocks":879,"cache_hit":0,"new_prefill":3065,"score_linear":49855.0,"score_lmetric":3256},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":45726,"ongoing_decode_tokens":30645,"pending_prefill_tokens":15081,"num_requests":1,"active_p_offloads":0,"cached_blocks":966,"cache_hit":0,"new_prefill":3065,"score_linear":45726.0,"score_lmetric":18146},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":540,"cache_hit":0,"new_prefill":3065,"score_linear":43607.0,"score_lmetric":3065},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":14944,"ongoing_decode_tokens":0,"pending_prefill_tokens":14944,"num_requests":1,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":3065,"score_linear":14944.0,"score_lmetric":18009},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":241141,"ongoing_decode_tokens":241141,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":512,"cache_hit":0,"new_prefill":3065,"score_linear":241141.0,"score_lmetric":9195}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348537.145739946,"t_first_token_unix":1779779402.1596835,"t_done":348537.375194789,"t_done_unix":1779779402.3891385},{"request_id":"1303462:1:1303462:274","session_id":"1303462","input_length":14944,"t_proxy_recv":348536.356311637,"t_decision_unix":1779779401.370253,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":14944,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":64654,"ongoing_decode_tokens":64654,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":594,"cache_hit":0,"new_prefill":14944,"score_linear":64654.0,"score_lmetric":29888},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":773,"cache_hit":0,"new_prefill":14944,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":768,"cache_hit":0,"new_prefill":14944,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":49855,"ongoing_decode_tokens":0,"pending_prefill_tokens":191,"num_requests":1,"active_p_offloads":0,"cached_blocks":879,"cache_hit":0,"new_prefill":14944,"score_linear":49855.0,"score_lmetric":15135},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":30645,"ongoing_decode_tokens":30645,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":966,"cache_hit":0,"new_prefill":14944,"score_linear":30645.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":540,"cache_hit":0,"new_prefill":14944,"score_linear":43607.0,"score_lmetric":14944},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":14944,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":241141,"ongoing_decode_tokens":241141,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":512,"cache_hit":0,"new_prefill":14944,"score_linear":241141.0,"score_lmetric":44832}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348537.685143128,"t_first_token_unix":1779779402.6990864,"t_done":348537.980805672,"t_done_unix":1779779402.9947486},{"request_id":"1303517:1:1303517:275","session_id":"1303517","input_length":15081,"t_proxy_recv":348536.519799242,"t_decision_unix":1779779401.5337405,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":15081,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":64654,"ongoing_decode_tokens":64654,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":594,"cache_hit":0,"new_prefill":15081,"score_linear":64654.0,"score_lmetric":30162},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":773,"cache_hit":0,"new_prefill":15081,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":768,"cache_hit":0,"new_prefill":15081,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":49855,"ongoing_decode_tokens":0,"pending_prefill_tokens":191,"num_requests":1,"active_p_offloads":0,"cached_blocks":879,"cache_hit":0,"new_prefill":15081,"score_linear":49855.0,"score_lmetric":15272},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":30645,"ongoing_decode_tokens":30645,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":966,"cache_hit":0,"new_prefill":15081,"score_linear":30645.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":540,"cache_hit":0,"new_prefill":15081,"score_linear":43607.0,"score_lmetric":15081},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":14944,"ongoing_decode_tokens":0,"pending_prefill_tokens":14944,"num_requests":1,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":15081,"score_linear":14944.0,"score_lmetric":30025},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":241141,"ongoing_decode_tokens":241141,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":512,"cache_hit":0,"new_prefill":15081,"score_linear":241141.0,"score_lmetric":45243}],"chosen_score_linear":30645.0,"chosen_score_lmetric":0,"t_first_token":348537.873645667,"t_first_token_unix":1779779402.8875895,"t_done":348538.372863203,"t_done_unix":1779779403.386807},{"request_id":"1302063:2:1303816:279","session_id":"1302063","input_length":30248,"t_proxy_recv":348537.6284386,"t_decision_unix":1779779402.64238,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":26112,"affinity_cache_ratio":0.8632636868553293,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":26112,"estimated_new_tokens":4136,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":64654,"ongoing_decode_tokens":64654,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":594,"cache_hit":0,"new_prefill":30248,"score_linear":64654.0,"score_lmetric":60496},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":778,"cache_hit":0,"new_prefill":30248,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":774,"cache_hit":26112,"new_prefill":4136,"score_linear":-26112.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":49855,"ongoing_decode_tokens":0,"pending_prefill_tokens":191,"num_requests":1,"active_p_offloads":0,"cached_blocks":879,"cache_hit":0,"new_prefill":30248,"score_linear":49855.0,"score_lmetric":30439},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":45726,"ongoing_decode_tokens":30645,"pending_prefill_tokens":15081,"num_requests":1,"active_p_offloads":0,"cached_blocks":966,"cache_hit":0,"new_prefill":30248,"score_linear":45726.0,"score_lmetric":45329},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":540,"cache_hit":0,"new_prefill":30248,"score_linear":43607.0,"score_lmetric":30248},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":14944,"ongoing_decode_tokens":0,"pending_prefill_tokens":14944,"num_requests":1,"active_p_offloads":0,"cached_blocks":765,"cache_hit":0,"new_prefill":30248,"score_linear":14944.0,"score_lmetric":45192},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":241141,"ongoing_decode_tokens":241141,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":512,"cache_hit":0,"new_prefill":30248,"score_linear":241141.0,"score_lmetric":90744}],"chosen_score_linear":-26112.0,"chosen_score_lmetric":0,"t_first_token":348538.478173373,"t_first_token_unix":1779779403.4921174,"t_done":348538.801297758,"t_done_unix":1779779403.815241},{"request_id":"1304239:1:1304239:280","session_id":"1304239","input_length":1300,"t_proxy_recv":348539.153075668,"t_decision_unix":1779779404.167017,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":1300,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":64654,"ongoing_decode_tokens":64654,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":594,"cache_hit":0,"new_prefill":1300,"score_linear":64654.0,"score_lmetric":2600},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":778,"cache_hit":0,"new_prefill":1300,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":782,"cache_hit":0,"new_prefill":1300,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":49855,"ongoing_decode_tokens":0,"pending_prefill_tokens":191,"num_requests":1,"active_p_offloads":0,"cached_blocks":879,"cache_hit":0,"new_prefill":1300,"score_linear":49855.0,"score_lmetric":1491},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":30645,"ongoing_decode_tokens":30645,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":995,"cache_hit":0,"new_prefill":1300,"score_linear":30645.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":540,"cache_hit":0,"new_prefill":1300,"score_linear":43607.0,"score_lmetric":1300},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":794,"cache_hit":0,"new_prefill":1300,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":241141,"ongoing_decode_tokens":241141,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":512,"cache_hit":0,"new_prefill":1300,"score_linear":241141.0,"score_lmetric":3900}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348539.250888009,"t_first_token_unix":1779779404.2648318,"t_done":348539.394649446,"t_done_unix":1779779404.4085927},{"request_id":"1294611:6:1302460:270","session_id":"1294611","input_length":30645,"t_proxy_recv":348535.10838257,"t_decision_unix":1779779400.1223242,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":25600,"affinity_cache_ratio":0.8353728177516724,"affinity_num_requests":-1,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":25600,"estimated_new_tokens":5045,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":64654,"ongoing_decode_tokens":0,"pending_prefill_tokens":31374,"num_requests":2,"active_p_offloads":0,"cached_blocks":594,"cache_hit":0,"new_prefill":30645,"score_linear":64654.0,"score_lmetric":124038},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":15958,"ongoing_decode_tokens":0,"pending_prefill_tokens":15958,"num_requests":1,"active_p_offloads":0,"cached_blocks":742,"cache_hit":0,"new_prefill":30645,"score_linear":15958.0,"score_lmetric":46603},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":768,"cache_hit":0,"new_prefill":30645,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":49855,"ongoing_decode_tokens":0,"pending_prefill_tokens":191,"num_requests":1,"active_p_offloads":0,"cached_blocks":879,"cache_hit":0,"new_prefill":30645,"score_linear":49855.0,"score_lmetric":30836},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":966,"cache_hit":25600,"new_prefill":5045,"score_linear":-25600.0,"score_lmetric":-5045},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":69555,"ongoing_decode_tokens":69555,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":538,"cache_hit":0,"new_prefill":30645,"score_linear":69555.0,"score_lmetric":61290},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":43181,"ongoing_decode_tokens":43181,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":762,"cache_hit":0,"new_prefill":30645,"score_linear":43181.0,"score_lmetric":30645},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":241141,"ongoing_decode_tokens":241141,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":512,"cache_hit":12800,"new_prefill":17845,"score_linear":228341.0,"score_lmetric":53535}],"chosen_score_linear":-25600.0,"chosen_score_lmetric":-5045,"t_first_token":348536.156992778,"t_first_token_unix":1779779401.170936,"t_done":348539.690077556,"t_done_unix":1779779404.704021},{"request_id":"1266668:7:1287468:197","session_id":"1266668","input_length":87741,"t_proxy_recv":348530.617094758,"t_decision_unix":1779779395.631036,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":80384,"affinity_cache_ratio":0.916150944256391,"affinity_num_requests":2,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":80384,"estimated_new_tokens":7357,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":594,"cache_hit":7680,"new_prefill":80061,"score_linear":-7680.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":740,"cache_hit":0,"new_prefill":87741,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":717,"cache_hit":0,"new_prefill":87741,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":867,"cache_hit":0,"new_prefill":87741,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":52835,"ongoing_decode_tokens":23863,"pending_prefill_tokens":18220,"num_requests":1,"active_p_offloads":0,"cached_blocks":927,"cache_hit":0,"new_prefill":87741,"score_linear":52835.0,"score_lmetric":105961},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":69555,"ongoing_decode_tokens":0,"pending_prefill_tokens":30643,"num_requests":2,"active_p_offloads":0,"cached_blocks":538,"cache_hit":0,"new_prefill":87741,"score_linear":69555.0,"score_lmetric":236768},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":43181,"ongoing_decode_tokens":43181,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":762,"cache_hit":7680,"new_prefill":80061,"score_linear":35501.0,"score_lmetric":80061},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":153400,"ongoing_decode_tokens":153400,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":512,"cache_hit":80384,"new_prefill":7357,"score_linear":73016.0,"score_lmetric":14714}],"chosen_score_linear":73016.0,"chosen_score_lmetric":14714,"t_first_token":348534.481760423,"t_first_token_unix":1779779399.4957035,"t_done":348542.175787747,"t_done_unix":1779779407.1897314},{"request_id":"1304239:2:1304866:281","session_id":"1304239","input_length":7798,"t_proxy_recv":348541.302558394,"t_decision_unix":1779779406.3164997,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":2,"chosen_idx":4,"affinity_cache_hit":1024,"affinity_cache_ratio":0.13131572197999486,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":-7798,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":7798,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":64654,"ongoing_decode_tokens":64654,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":594,"cache_hit":0,"new_prefill":7798,"score_linear":64654.0,"score_lmetric":15596},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":778,"cache_hit":0,"new_prefill":7798,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":784,"cache_hit":1024,"new_prefill":6774,"score_linear":-1024.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":49855,"ongoing_decode_tokens":0,"pending_prefill_tokens":191,"num_requests":1,"active_p_offloads":0,"cached_blocks":879,"cache_hit":0,"new_prefill":7798,"score_linear":49855.0,"score_lmetric":7989},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1005,"cache_hit":0,"new_prefill":7798,"score_linear":0.0,"score_lmetric":-7798},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":540,"cache_hit":0,"new_prefill":7798,"score_linear":43607.0,"score_lmetric":7798},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":794,"cache_hit":0,"new_prefill":7798,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":241141,"ongoing_decode_tokens":241141,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":512,"cache_hit":0,"new_prefill":7798,"score_linear":241141.0,"score_lmetric":23394}],"chosen_score_linear":0.0,"chosen_score_lmetric":-7798,"t_first_token":348541.841111285,"t_first_token_unix":1779779406.8550546,"t_done":348542.234398627,"t_done_unix":1779779407.2483416},{"request_id":"1253743:3:1302294:268","session_id":"1253743","input_length":49855,"t_proxy_recv":348532.341799908,"t_decision_unix":1779779397.3557408,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":49664,"affinity_cache_ratio":0.996168889780363,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":49664,"estimated_new_tokens":191,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":34706,"ongoing_decode_tokens":0,"pending_prefill_tokens":31122,"num_requests":1,"active_p_offloads":0,"cached_blocks":594,"cache_hit":0,"new_prefill":49855,"score_linear":34706.0,"score_lmetric":80977},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":740,"cache_hit":4608,"new_prefill":45247,"score_linear":-4608.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":26441,"ongoing_decode_tokens":0,"pending_prefill_tokens":26441,"num_requests":1,"active_p_offloads":0,"cached_blocks":717,"cache_hit":0,"new_prefill":49855,"score_linear":26441.0,"score_lmetric":76296},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":879,"cache_hit":49664,"new_prefill":191,"score_linear":-49664.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":52835,"ongoing_decode_tokens":23863,"pending_prefill_tokens":18220,"num_requests":1,"active_p_offloads":0,"cached_blocks":927,"cache_hit":0,"new_prefill":49855,"score_linear":52835.0,"score_lmetric":68075},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":69555,"ongoing_decode_tokens":0,"pending_prefill_tokens":30643,"num_requests":2,"active_p_offloads":0,"cached_blocks":538,"cache_hit":17408,"new_prefill":32447,"score_linear":52147.0,"score_lmetric":126180},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":43181,"ongoing_decode_tokens":43181,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":762,"cache_hit":0,"new_prefill":49855,"score_linear":43181.0,"score_lmetric":49855},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":241141,"ongoing_decode_tokens":153400,"pending_prefill_tokens":7357,"num_requests":3,"active_p_offloads":0,"cached_blocks":512,"cache_hit":0,"new_prefill":49855,"score_linear":241141.0,"score_lmetric":171636}],"chosen_score_linear":-49664.0,"chosen_score_lmetric":0,"t_first_token":348541.664042303,"t_first_token_unix":1779779406.6779861,"t_done":348542.255300506,"t_done_unix":1779779407.2692442},{"request_id":"1304943:1:1304943:283","session_id":"1304943","input_length":2772,"t_proxy_recv":348541.553983666,"t_decision_unix":1779779406.567925,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":2772,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":64654,"ongoing_decode_tokens":64654,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":594,"cache_hit":0,"new_prefill":2772,"score_linear":64654.0,"score_lmetric":5544},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":778,"cache_hit":0,"new_prefill":2772,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":784,"cache_hit":0,"new_prefill":2772,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":49855,"ongoing_decode_tokens":0,"pending_prefill_tokens":191,"num_requests":1,"active_p_offloads":0,"cached_blocks":879,"cache_hit":0,"new_prefill":2772,"score_linear":49855.0,"score_lmetric":2963},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":7798,"ongoing_decode_tokens":0,"pending_prefill_tokens":7798,"num_requests":0,"active_p_offloads":0,"cached_blocks":1005,"cache_hit":0,"new_prefill":2772,"score_linear":7798.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":540,"cache_hit":0,"new_prefill":2772,"score_linear":43607.0,"score_lmetric":2772},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":794,"cache_hit":0,"new_prefill":2772,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":241141,"ongoing_decode_tokens":241141,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":512,"cache_hit":0,"new_prefill":2772,"score_linear":241141.0,"score_lmetric":8316}],"chosen_score_linear":7798.0,"chosen_score_lmetric":0,"t_first_token":348542.003067789,"t_first_token_unix":1779779407.0170112,"t_done":348542.487922878,"t_done_unix":1779779407.501866},{"request_id":"1294611:6:1305080:284","session_id":"1294611","input_length":30995,"t_proxy_recv":348542.063873009,"t_decision_unix":1779779407.0778143,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":30720,"affinity_cache_ratio":0.9911276012260042,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":30720,"estimated_new_tokens":275,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":64654,"ongoing_decode_tokens":64654,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":594,"cache_hit":0,"new_prefill":30995,"score_linear":64654.0,"score_lmetric":61990},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":778,"cache_hit":0,"new_prefill":30995,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":784,"cache_hit":0,"new_prefill":30995,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":49855,"ongoing_decode_tokens":49855,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":879,"cache_hit":0,"new_prefill":30995,"score_linear":49855.0,"score_lmetric":30995},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":10570,"ongoing_decode_tokens":10570,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1005,"cache_hit":30720,"new_prefill":275,"score_linear":-20150.0,"score_lmetric":275},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":540,"cache_hit":0,"new_prefill":30995,"score_linear":43607.0,"score_lmetric":30995},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":794,"cache_hit":0,"new_prefill":30995,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":241141,"ongoing_decode_tokens":241141,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":512,"cache_hit":12800,"new_prefill":18195,"score_linear":228341.0,"score_lmetric":54585}],"chosen_score_linear":-20150.0,"chosen_score_lmetric":275,"t_first_token":348542.133099103,"t_first_token_unix":1779779407.147042,"t_done":348543.52792404,"t_done_unix":1779779408.5418673},{"request_id":"1303651:2:1305707:287","session_id":"1303651","input_length":12795,"t_proxy_recv":348544.496185157,"t_decision_unix":1779779409.5101266,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":2,"chosen_idx":4,"affinity_cache_hit":3072,"affinity_cache_ratio":0.24009378663540445,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":-12795,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":12795,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":125945,"ongoing_decode_tokens":64654,"pending_prefill_tokens":875,"num_requests":3,"active_p_offloads":0,"cached_blocks":594,"cache_hit":0,"new_prefill":12795,"score_linear":125945.0,"score_lmetric":41010},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":75826,"ongoing_decode_tokens":0,"pending_prefill_tokens":75826,"num_requests":1,"active_p_offloads":0,"cached_blocks":778,"cache_hit":0,"new_prefill":12795,"score_linear":75826.0,"score_lmetric":88621},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":784,"cache_hit":3072,"new_prefill":9723,"score_linear":-3072.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":879,"cache_hit":0,"new_prefill":12795,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1025,"cache_hit":0,"new_prefill":12795,"score_linear":0.0,"score_lmetric":-12795},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":540,"cache_hit":0,"new_prefill":12795,"score_linear":43607.0,"score_lmetric":12795},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":794,"cache_hit":0,"new_prefill":12795,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":249505,"ongoing_decode_tokens":153400,"pending_prefill_tokens":8041,"num_requests":3,"active_p_offloads":0,"cached_blocks":527,"cache_hit":0,"new_prefill":12795,"score_linear":249505.0,"score_lmetric":62508}],"chosen_score_linear":0.0,"chosen_score_lmetric":-12795,"t_first_token":348545.56333151,"t_first_token_unix":1779779410.5772834,"t_done":348545.685057191,"t_done_unix":1779779410.6990006},{"request_id":"1279412:9:1305633:286","session_id":"1279412","input_length":61291,"t_proxy_recv":348544.232366627,"t_decision_unix":1779779409.246308,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":60416,"affinity_cache_ratio":0.985723841999641,"affinity_num_requests":2,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":60416,"estimated_new_tokens":875,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":64654,"ongoing_decode_tokens":64654,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":594,"cache_hit":60416,"new_prefill":875,"score_linear":4238.0,"score_lmetric":1750},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":75826,"ongoing_decode_tokens":0,"pending_prefill_tokens":75826,"num_requests":1,"active_p_offloads":0,"cached_blocks":778,"cache_hit":0,"new_prefill":61291,"score_linear":75826.0,"score_lmetric":137117},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":784,"cache_hit":0,"new_prefill":61291,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":879,"cache_hit":0,"new_prefill":61291,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1025,"cache_hit":0,"new_prefill":61291,"score_linear":0.0,"score_lmetric":-61291},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":540,"cache_hit":0,"new_prefill":61291,"score_linear":43607.0,"score_lmetric":61291},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":794,"cache_hit":12800,"new_prefill":48491,"score_linear":-12800.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":249505,"ongoing_decode_tokens":153400,"pending_prefill_tokens":8041,"num_requests":3,"active_p_offloads":0,"cached_blocks":527,"cache_hit":12800,"new_prefill":48491,"score_linear":236705.0,"score_lmetric":169596}],"chosen_score_linear":4238.0,"chosen_score_lmetric":1750,"t_first_token":348544.552851264,"t_first_token_unix":1779779409.566796,"t_done":348545.955188867,"t_done_unix":1779779410.969133},{"request_id":"1303462:2:1306126:290","session_id":"1303462","input_length":18631,"t_proxy_recv":348545.724429403,"t_decision_unix":1779779410.738371,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":14848,"affinity_cache_ratio":0.7969513176963126,"affinity_num_requests":1,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":14848,"estimated_new_tokens":3783,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":125945,"ongoing_decode_tokens":125945,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":594,"cache_hit":0,"new_prefill":18631,"score_linear":125945.0,"score_lmetric":55893},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":75826,"ongoing_decode_tokens":0,"pending_prefill_tokens":75826,"num_requests":1,"active_p_offloads":0,"cached_blocks":778,"cache_hit":0,"new_prefill":18631,"score_linear":75826.0,"score_lmetric":94457},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":14027,"ongoing_decode_tokens":0,"pending_prefill_tokens":14027,"num_requests":1,"active_p_offloads":0,"cached_blocks":784,"cache_hit":0,"new_prefill":18631,"score_linear":14027.0,"score_lmetric":32658},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":879,"cache_hit":0,"new_prefill":18631,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1050,"cache_hit":0,"new_prefill":18631,"score_linear":0.0,"score_lmetric":-18631},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":540,"cache_hit":0,"new_prefill":18631,"score_linear":43607.0,"score_lmetric":18631},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":18915,"ongoing_decode_tokens":18915,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":794,"cache_hit":14848,"new_prefill":3783,"score_linear":4067.0,"score_lmetric":3783},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":249505,"ongoing_decode_tokens":153400,"pending_prefill_tokens":8041,"num_requests":3,"active_p_offloads":0,"cached_blocks":527,"cache_hit":0,"new_prefill":18631,"score_linear":249505.0,"score_lmetric":80016}],"chosen_score_linear":4067.0,"chosen_score_lmetric":3783,"t_first_token":348546.276572569,"t_first_token_unix":1779779411.2905161,"t_done":348546.325786329,"t_done_unix":1779779411.339729},{"request_id":"1299507:2:1305770:288","session_id":"1299507","input_length":18915,"t_proxy_recv":348544.691436492,"t_decision_unix":1779779409.7053773,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":18432,"affinity_cache_ratio":0.9744647105471848,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":18432,"estimated_new_tokens":483,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":125945,"ongoing_decode_tokens":125945,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":594,"cache_hit":0,"new_prefill":18915,"score_linear":125945.0,"score_lmetric":56745},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":75826,"ongoing_decode_tokens":0,"pending_prefill_tokens":75826,"num_requests":1,"active_p_offloads":0,"cached_blocks":778,"cache_hit":0,"new_prefill":18915,"score_linear":75826.0,"score_lmetric":94741},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":784,"cache_hit":0,"new_prefill":18915,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":879,"cache_hit":0,"new_prefill":18915,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":12795,"ongoing_decode_tokens":0,"pending_prefill_tokens":12795,"num_requests":0,"active_p_offloads":0,"cached_blocks":1025,"cache_hit":0,"new_prefill":18915,"score_linear":12795.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":540,"cache_hit":0,"new_prefill":18915,"score_linear":43607.0,"score_lmetric":18915},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":794,"cache_hit":18432,"new_prefill":483,"score_linear":-18432.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":249505,"ongoing_decode_tokens":153400,"pending_prefill_tokens":8041,"num_requests":3,"active_p_offloads":0,"cached_blocks":527,"cache_hit":0,"new_prefill":18915,"score_linear":249505.0,"score_lmetric":80868}],"chosen_score_linear":-18432.0,"chosen_score_lmetric":0,"t_first_token":348544.739056241,"t_first_token_unix":1779779409.7529998,"t_done":348546.597592237,"t_done_unix":1779779411.6115358},{"request_id":"1306347:1:1306347:291","session_id":"1306347","input_length":897,"t_proxy_recv":348546.512074594,"t_decision_unix":1779779411.5260158,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":-897,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":897,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":64654,"ongoing_decode_tokens":64654,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":595,"cache_hit":0,"new_prefill":897,"score_linear":64654.0,"score_lmetric":1794},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":75826,"ongoing_decode_tokens":0,"pending_prefill_tokens":75826,"num_requests":1,"active_p_offloads":0,"cached_blocks":778,"cache_hit":0,"new_prefill":897,"score_linear":75826.0,"score_lmetric":76723},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":14027,"ongoing_decode_tokens":14027,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":784,"cache_hit":0,"new_prefill":897,"score_linear":14027.0,"score_lmetric":897},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":879,"cache_hit":0,"new_prefill":897,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1050,"cache_hit":0,"new_prefill":897,"score_linear":0.0,"score_lmetric":-897},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":540,"cache_hit":0,"new_prefill":897,"score_linear":43607.0,"score_lmetric":897},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":18915,"ongoing_decode_tokens":18915,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":801,"cache_hit":0,"new_prefill":897,"score_linear":18915.0,"score_lmetric":897},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":249505,"ongoing_decode_tokens":153400,"pending_prefill_tokens":8041,"num_requests":3,"active_p_offloads":0,"cached_blocks":527,"cache_hit":0,"new_prefill":897,"score_linear":249505.0,"score_lmetric":26814}],"chosen_score_linear":0.0,"chosen_score_lmetric":-897,"t_first_token":348546.584640831,"t_first_token_unix":1779779411.5985844,"t_done":348546.958739147,"t_done_unix":1779779411.9726822},{"request_id":"1269373:6:1285668:189","session_id":"1269373","input_length":38788,"t_proxy_recv":348528.563214596,"t_decision_unix":1779779393.577156,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":30720,"affinity_cache_ratio":0.7919975250077343,"affinity_num_requests":2,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":30720,"estimated_new_tokens":8068,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":594,"cache_hit":7680,"new_prefill":31108,"score_linear":-7680.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":740,"cache_hit":0,"new_prefill":38788,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":717,"cache_hit":0,"new_prefill":38788,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":867,"cache_hit":0,"new_prefill":38788,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":83370,"ongoing_decode_tokens":83370,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":811,"cache_hit":0,"new_prefill":38788,"score_linear":83370.0,"score_lmetric":38788},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":69555,"ongoing_decode_tokens":0,"pending_prefill_tokens":30643,"num_requests":2,"active_p_offloads":0,"cached_blocks":538,"cache_hit":0,"new_prefill":38788,"score_linear":69555.0,"score_lmetric":138862},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":18893,"ongoing_decode_tokens":18893,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":747,"cache_hit":7680,"new_prefill":31108,"score_linear":11213.0,"score_lmetric":31108},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":194667,"ongoing_decode_tokens":194667,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":469,"cache_hit":30720,"new_prefill":8068,"score_linear":163947.0,"score_lmetric":16136}],"chosen_score_linear":163947.0,"chosen_score_lmetric":16136,"t_first_token":348530.479334487,"t_first_token_unix":1779779395.4932778,"t_done":348547.090036626,"t_done_unix":1779779412.1039803},{"request_id":"1305906:1:1305906:289","session_id":"1305906","input_length":14027,"t_proxy_recv":348545.110559113,"t_decision_unix":1779779410.1245005,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":14027,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":125945,"ongoing_decode_tokens":125945,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":594,"cache_hit":0,"new_prefill":14027,"score_linear":125945.0,"score_lmetric":42081},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":75826,"ongoing_decode_tokens":0,"pending_prefill_tokens":75826,"num_requests":1,"active_p_offloads":0,"cached_blocks":778,"cache_hit":0,"new_prefill":14027,"score_linear":75826.0,"score_lmetric":89853},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":784,"cache_hit":0,"new_prefill":14027,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":879,"cache_hit":0,"new_prefill":14027,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":12795,"ongoing_decode_tokens":0,"pending_prefill_tokens":12795,"num_requests":0,"active_p_offloads":0,"cached_blocks":1025,"cache_hit":0,"new_prefill":14027,"score_linear":12795.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":540,"cache_hit":0,"new_prefill":14027,"score_linear":43607.0,"score_lmetric":14027},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":18915,"ongoing_decode_tokens":18915,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":794,"cache_hit":0,"new_prefill":14027,"score_linear":18915.0,"score_lmetric":14027},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":249505,"ongoing_decode_tokens":153400,"pending_prefill_tokens":8041,"num_requests":3,"active_p_offloads":0,"cached_blocks":527,"cache_hit":0,"new_prefill":14027,"score_linear":249505.0,"score_lmetric":66204}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348546.316207721,"t_first_token_unix":1779779411.3301506,"t_done":348547.365624034,"t_done_unix":1779779412.3795676},{"request_id":"1275274:1:1275274:136","session_id":"1275274","input_length":114612,"t_proxy_recv":348439.270625266,"t_decision_unix":1779779304.284566,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":114612,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135235,"ongoing_decode_tokens":0,"pending_prefill_tokens":113219,"num_requests":2,"active_p_offloads":0,"cached_blocks":156,"cache_hit":0,"new_prefill":114612,"score_linear":135235.0,"score_lmetric":455662},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102074,"ongoing_decode_tokens":0,"pending_prefill_tokens":98490,"num_requests":1,"active_p_offloads":0,"cached_blocks":265,"cache_hit":0,"new_prefill":114612,"score_linear":102074.0,"score_lmetric":213102},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":150574,"ongoing_decode_tokens":0,"pending_prefill_tokens":117806,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":114612,"score_linear":150574.0,"score_lmetric":464836},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":108713,"ongoing_decode_tokens":15283,"pending_prefill_tokens":93430,"num_requests":2,"active_p_offloads":0,"cached_blocks":358,"cache_hit":0,"new_prefill":114612,"score_linear":108713.0,"score_lmetric":416084},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":49602,"ongoing_decode_tokens":49602,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":404,"cache_hit":0,"new_prefill":114612,"score_linear":49602.0,"score_lmetric":114612},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":299,"cache_hit":0,"new_prefill":114612,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":489,"cache_hit":0,"new_prefill":114612,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":388,"cache_hit":0,"new_prefill":114612,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348481.481446313,"t_first_token_unix":1779779346.4953897,"t_done":348548.841174523,"t_done_unix":1779779413.8551178},{"request_id":"1266668:8:1292408:218","session_id":"1266668","input_length":96105,"t_proxy_recv":348542.195018754,"t_decision_unix":1779779407.20896,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":88064,"affinity_cache_ratio":0.916331096196868,"affinity_num_requests":2,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":88064,"estimated_new_tokens":8041,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":64654,"ongoing_decode_tokens":64654,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":594,"cache_hit":7680,"new_prefill":88425,"score_linear":56974.0,"score_lmetric":176850},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":778,"cache_hit":0,"new_prefill":96105,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":784,"cache_hit":0,"new_prefill":96105,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":49855,"ongoing_decode_tokens":49855,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":879,"cache_hit":0,"new_prefill":96105,"score_linear":49855.0,"score_lmetric":96105},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":41565,"ongoing_decode_tokens":41565,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1005,"cache_hit":0,"new_prefill":96105,"score_linear":41565.0,"score_lmetric":192210},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":540,"cache_hit":0,"new_prefill":96105,"score_linear":43607.0,"score_lmetric":96105},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":794,"cache_hit":7680,"new_prefill":88425,"score_linear":-7680.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":153400,"ongoing_decode_tokens":153400,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":527,"cache_hit":88064,"new_prefill":8041,"score_linear":65336.0,"score_lmetric":16082}],"chosen_score_linear":65336.0,"chosen_score_lmetric":16082,"t_first_token":348546.786085652,"t_first_token_unix":1779779411.8000295,"t_done":348549.373750573,"t_done_unix":1779779414.3876944},{"request_id":"1253743:4:1307058:295","session_id":"1253743","input_length":49975,"t_proxy_recv":348549.067438663,"t_decision_unix":1779779414.08138,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":49664,"affinity_cache_ratio":0.9937768884442221,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":49664,"estimated_new_tokens":311,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":64654,"ongoing_decode_tokens":64654,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":595,"cache_hit":0,"new_prefill":49975,"score_linear":64654.0,"score_lmetric":99950},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":75826,"ongoing_decode_tokens":0,"pending_prefill_tokens":75826,"num_requests":1,"active_p_offloads":0,"cached_blocks":778,"cache_hit":4608,"new_prefill":45367,"score_linear":71218.0,"score_lmetric":121193},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":811,"cache_hit":0,"new_prefill":49975,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":879,"cache_hit":49664,"new_prefill":311,"score_linear":-49664.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":57218,"ongoing_decode_tokens":0,"pending_prefill_tokens":18818,"num_requests":1,"active_p_offloads":0,"cached_blocks":1051,"cache_hit":0,"new_prefill":49975,"score_linear":57218.0,"score_lmetric":68793},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":74665,"ongoing_decode_tokens":74665,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":540,"cache_hit":17408,"new_prefill":32567,"score_linear":57257.0,"score_lmetric":65134},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":801,"cache_hit":0,"new_prefill":49975,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":138850,"ongoing_decode_tokens":138850,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":768,"cache_hit":0,"new_prefill":49975,"score_linear":138850.0,"score_lmetric":99950}],"chosen_score_linear":-49664.0,"chosen_score_lmetric":0,"t_first_token":348549.185654262,"t_first_token_unix":1779779414.1995976,"t_done":348549.640579996,"t_done_unix":1779779414.6545231},{"request_id":"1266668:9:1295576:231","session_id":"1266668","input_length":96717,"t_proxy_recv":348549.38713017,"t_decision_unix":1779779414.4010715,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":95744,"affinity_cache_ratio":0.9899397210418024,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":95744,"estimated_new_tokens":973,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":64654,"ongoing_decode_tokens":64654,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":595,"cache_hit":7680,"new_prefill":89037,"score_linear":56974.0,"score_lmetric":178074},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":75826,"ongoing_decode_tokens":0,"pending_prefill_tokens":75826,"num_requests":1,"active_p_offloads":0,"cached_blocks":778,"cache_hit":0,"new_prefill":96717,"score_linear":75826.0,"score_lmetric":172543},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":811,"cache_hit":0,"new_prefill":96717,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":49975,"ongoing_decode_tokens":49975,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":879,"cache_hit":0,"new_prefill":96717,"score_linear":49975.0,"score_lmetric":96717},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":57218,"ongoing_decode_tokens":0,"pending_prefill_tokens":18818,"num_requests":1,"active_p_offloads":0,"cached_blocks":1051,"cache_hit":0,"new_prefill":96717,"score_linear":57218.0,"score_lmetric":115535},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":74665,"ongoing_decode_tokens":74665,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":540,"cache_hit":0,"new_prefill":96717,"score_linear":74665.0,"score_lmetric":193434},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":801,"cache_hit":7680,"new_prefill":89037,"score_linear":-7680.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":42745,"ongoing_decode_tokens":42745,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":783,"cache_hit":95744,"new_prefill":973,"score_linear":-52999.0,"score_lmetric":973}],"chosen_score_linear":-52999.0,"chosen_score_lmetric":973,"t_first_token":348549.872835753,"t_first_token_unix":1779779414.8867793,"t_done":348550.262164105,"t_done_unix":1779779415.2761075},{"request_id":"1277428:2:1306373:292","session_id":"1277428","input_length":31058,"t_proxy_recv":348546.623333974,"t_decision_unix":1779779411.6372757,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":27136,"affinity_cache_ratio":0.8737201365187713,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":27136,"estimated_new_tokens":3922,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":64654,"ongoing_decode_tokens":64654,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":595,"cache_hit":0,"new_prefill":31058,"score_linear":64654.0,"score_lmetric":62116},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":75826,"ongoing_decode_tokens":0,"pending_prefill_tokens":75826,"num_requests":1,"active_p_offloads":0,"cached_blocks":778,"cache_hit":4608,"new_prefill":26450,"score_linear":71218.0,"score_lmetric":102276},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":14027,"ongoing_decode_tokens":14027,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":784,"cache_hit":0,"new_prefill":31058,"score_linear":14027.0,"score_lmetric":31058},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":879,"cache_hit":17408,"new_prefill":13650,"score_linear":-17408.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":897,"ongoing_decode_tokens":897,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1050,"cache_hit":0,"new_prefill":31058,"score_linear":897.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":540,"cache_hit":27136,"new_prefill":3922,"score_linear":16471.0,"score_lmetric":3922},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":801,"cache_hit":0,"new_prefill":31058,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":249505,"ongoing_decode_tokens":153400,"pending_prefill_tokens":8041,"num_requests":3,"active_p_offloads":0,"cached_blocks":527,"cache_hit":0,"new_prefill":31058,"score_linear":249505.0,"score_lmetric":117297}],"chosen_score_linear":16471.0,"chosen_score_lmetric":3922,"t_first_token":348547.49740117,"t_first_token_unix":1779779412.5113447,"t_done":348552.609686381,"t_done_unix":1779779417.6236298},{"request_id":"1272313:11:1302571:271","session_id":"1272313","input_length":29948,"t_proxy_recv":348533.28703773,"t_decision_unix":1779779398.3009794,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":29696,"affinity_cache_ratio":0.991585414718846,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":29696,"estimated_new_tokens":252,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":34706,"ongoing_decode_tokens":0,"pending_prefill_tokens":31122,"num_requests":1,"active_p_offloads":0,"cached_blocks":594,"cache_hit":29696,"new_prefill":252,"score_linear":5010.0,"score_lmetric":31374},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":1105,"ongoing_decode_tokens":1105,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":740,"cache_hit":0,"new_prefill":29948,"score_linear":1105.0,"score_lmetric":29948},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":26441,"ongoing_decode_tokens":0,"pending_prefill_tokens":26441,"num_requests":1,"active_p_offloads":0,"cached_blocks":717,"cache_hit":0,"new_prefill":29948,"score_linear":26441.0,"score_lmetric":56389},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":49855,"ongoing_decode_tokens":0,"pending_prefill_tokens":191,"num_requests":1,"active_p_offloads":0,"cached_blocks":879,"cache_hit":0,"new_prefill":29948,"score_linear":49855.0,"score_lmetric":30139},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":23863,"ongoing_decode_tokens":23863,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":962,"cache_hit":0,"new_prefill":29948,"score_linear":23863.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":69555,"ongoing_decode_tokens":69555,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":538,"cache_hit":0,"new_prefill":29948,"score_linear":69555.0,"score_lmetric":59896},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":43181,"ongoing_decode_tokens":43181,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":762,"cache_hit":7680,"new_prefill":22268,"score_linear":35501.0,"score_lmetric":22268},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":241141,"ongoing_decode_tokens":153400,"pending_prefill_tokens":7357,"num_requests":3,"active_p_offloads":0,"cached_blocks":512,"cache_hit":10752,"new_prefill":19196,"score_linear":230389.0,"score_lmetric":79659}],"chosen_score_linear":5010.0,"chosen_score_lmetric":31374,"t_first_token":348535.75179813,"t_first_token_unix":1779779400.7657416,"t_done":348553.186232327,"t_done_unix":1779779418.2001755},{"request_id":"1302449:3:1308121:299","session_id":"1302449","input_length":8729,"t_proxy_recv":348552.802172345,"t_decision_unix":1779779417.8161137,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":1,"chosen_idx":3,"affinity_cache_hit":3584,"affinity_cache_ratio":0.41058540497193263,"affinity_num_requests":1,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":8729,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":126159,"ongoing_decode_tokens":64654,"pending_prefill_tokens":577,"num_requests":3,"active_p_offloads":0,"cached_blocks":595,"cache_hit":0,"new_prefill":8729,"score_linear":126159.0,"score_lmetric":27918},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":75826,"ongoing_decode_tokens":0,"pending_prefill_tokens":75826,"num_requests":1,"active_p_offloads":0,"cached_blocks":778,"cache_hit":3584,"new_prefill":5145,"score_linear":72242.0,"score_lmetric":80971},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":811,"cache_hit":0,"new_prefill":8729,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":879,"cache_hit":0,"new_prefill":8729,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":80349,"ongoing_decode_tokens":57218,"pending_prefill_tokens":8283,"num_requests":2,"active_p_offloads":0,"cached_blocks":1051,"cache_hit":0,"new_prefill":8729,"score_linear":80349.0,"score_lmetric":34024},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":548,"cache_hit":0,"new_prefill":8729,"score_linear":43607.0,"score_lmetric":8729},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":801,"cache_hit":0,"new_prefill":8729,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":139890,"ongoing_decode_tokens":139890,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":784,"cache_hit":0,"new_prefill":8729,"score_linear":139890.0,"score_lmetric":17458}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348553.451373479,"t_first_token_unix":1779779418.4653165,"t_done":348553.536962581,"t_done_unix":1779779418.550906},{"request_id":"1279412:10:1308091:298","session_id":"1279412","input_length":61505,"t_proxy_recv":348552.703131049,"t_decision_unix":1779779417.7170727,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":60928,"affinity_cache_ratio":0.9906186488903341,"affinity_num_requests":2,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":60928,"estimated_new_tokens":577,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":64654,"ongoing_decode_tokens":64654,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":595,"cache_hit":60928,"new_prefill":577,"score_linear":3726.0,"score_lmetric":1154},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":75826,"ongoing_decode_tokens":0,"pending_prefill_tokens":75826,"num_requests":1,"active_p_offloads":0,"cached_blocks":778,"cache_hit":0,"new_prefill":61505,"score_linear":75826.0,"score_lmetric":137331},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":811,"cache_hit":0,"new_prefill":61505,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":879,"cache_hit":0,"new_prefill":61505,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":80349,"ongoing_decode_tokens":57218,"pending_prefill_tokens":8283,"num_requests":2,"active_p_offloads":0,"cached_blocks":1051,"cache_hit":0,"new_prefill":61505,"score_linear":80349.0,"score_lmetric":139576},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":548,"cache_hit":0,"new_prefill":61505,"score_linear":43607.0,"score_lmetric":61505},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":801,"cache_hit":12800,"new_prefill":48705,"score_linear":-12800.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":139890,"ongoing_decode_tokens":139890,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":784,"cache_hit":12800,"new_prefill":48705,"score_linear":127090.0,"score_lmetric":97410}],"chosen_score_linear":3726.0,"chosen_score_lmetric":1154,"t_first_token":348552.887387309,"t_first_token_unix":1779779417.9013305,"t_done":348553.980653884,"t_done_unix":1779779418.9945972},{"request_id":"1266668:10:1297641:241","session_id":"1266668","input_length":97145,"t_proxy_recv":348550.28089483,"t_decision_unix":1779779415.2948365,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":96256,"affinity_cache_ratio":0.9908487312779866,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":96256,"estimated_new_tokens":889,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":64654,"ongoing_decode_tokens":64654,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":595,"cache_hit":7680,"new_prefill":89465,"score_linear":56974.0,"score_lmetric":178930},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":75826,"ongoing_decode_tokens":0,"pending_prefill_tokens":75826,"num_requests":1,"active_p_offloads":0,"cached_blocks":778,"cache_hit":0,"new_prefill":97145,"score_linear":75826.0,"score_lmetric":172971},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":811,"cache_hit":0,"new_prefill":97145,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":879,"cache_hit":0,"new_prefill":97145,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":57218,"ongoing_decode_tokens":0,"pending_prefill_tokens":18818,"num_requests":1,"active_p_offloads":0,"cached_blocks":1051,"cache_hit":0,"new_prefill":97145,"score_linear":57218.0,"score_lmetric":115963},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":74665,"ongoing_decode_tokens":74665,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":540,"cache_hit":0,"new_prefill":97145,"score_linear":74665.0,"score_lmetric":194290},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":801,"cache_hit":7680,"new_prefill":89465,"score_linear":-7680.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":42745,"ongoing_decode_tokens":42745,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":784,"cache_hit":96256,"new_prefill":889,"score_linear":-53511.0,"score_lmetric":889}],"chosen_score_linear":-53511.0,"chosen_score_lmetric":889,"t_first_token":348550.757799799,"t_first_token_unix":1779779415.771744,"t_done":348554.168404055,"t_done_unix":1779779419.1823473},{"request_id":"1303651:3:1308202:300","session_id":"1303651","input_length":22896,"t_proxy_recv":348553.056282265,"t_decision_unix":1779779418.0702236,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":12800,"affinity_cache_ratio":0.5590496156533893,"affinity_num_requests":2,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":12800,"estimated_new_tokens":10096,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":126159,"ongoing_decode_tokens":126159,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":595,"cache_hit":0,"new_prefill":22896,"score_linear":126159.0,"score_lmetric":68688},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":75826,"ongoing_decode_tokens":0,"pending_prefill_tokens":75826,"num_requests":1,"active_p_offloads":0,"cached_blocks":778,"cache_hit":0,"new_prefill":22896,"score_linear":75826.0,"score_lmetric":98722},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":811,"cache_hit":3072,"new_prefill":19824,"score_linear":-3072.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":8729,"ongoing_decode_tokens":0,"pending_prefill_tokens":8729,"num_requests":1,"active_p_offloads":0,"cached_blocks":879,"cache_hit":0,"new_prefill":22896,"score_linear":8729.0,"score_lmetric":31625},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":80349,"ongoing_decode_tokens":57218,"pending_prefill_tokens":8283,"num_requests":2,"active_p_offloads":0,"cached_blocks":1051,"cache_hit":12800,"new_prefill":10096,"score_linear":67549.0,"score_lmetric":36758},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":548,"cache_hit":0,"new_prefill":22896,"score_linear":43607.0,"score_lmetric":22896},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":801,"cache_hit":0,"new_prefill":22896,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":139890,"ongoing_decode_tokens":139890,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":784,"cache_hit":0,"new_prefill":22896,"score_linear":139890.0,"score_lmetric":45792}],"chosen_score_linear":67549.0,"chosen_score_lmetric":36758,"t_first_token":348554.903387046,"t_first_token_unix":1779779419.9173307,"t_done":348555.076027374,"t_done_unix":1779779420.0899706},{"request_id":"1304239:3:1307007:294","session_id":"1304239","input_length":13840,"t_proxy_recv":348548.866939326,"t_decision_unix":1779779413.8808804,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":7680,"affinity_cache_ratio":0.5549132947976878,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":7680,"estimated_new_tokens":6160,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":64654,"ongoing_decode_tokens":64654,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":595,"cache_hit":0,"new_prefill":13840,"score_linear":64654.0,"score_lmetric":27680},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":75826,"ongoing_decode_tokens":0,"pending_prefill_tokens":75826,"num_requests":1,"active_p_offloads":0,"cached_blocks":778,"cache_hit":0,"new_prefill":13840,"score_linear":75826.0,"score_lmetric":89666},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":811,"cache_hit":1024,"new_prefill":12816,"score_linear":-1024.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":879,"cache_hit":0,"new_prefill":13840,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":43378,"ongoing_decode_tokens":0,"pending_prefill_tokens":12658,"num_requests":0,"active_p_offloads":0,"cached_blocks":1051,"cache_hit":7680,"new_prefill":6160,"score_linear":35698.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":74665,"ongoing_decode_tokens":74665,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":540,"cache_hit":0,"new_prefill":13840,"score_linear":74665.0,"score_lmetric":27680},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":801,"cache_hit":0,"new_prefill":13840,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":138850,"ongoing_decode_tokens":138850,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":768,"cache_hit":0,"new_prefill":13840,"score_linear":138850.0,"score_lmetric":27680}],"chosen_score_linear":35698.0,"chosen_score_lmetric":0,"t_first_token":348552.214705876,"t_first_token_unix":1779779417.2286491,"t_done":348555.230868088,"t_done_unix":1779779420.2448113},{"request_id":"1303517:2:1307703:297","session_id":"1303517","input_length":23131,"t_proxy_recv":348551.392936694,"t_decision_unix":1779779416.406878,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":14848,"affinity_cache_ratio":0.6419091262807488,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":14848,"estimated_new_tokens":8283,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":64654,"ongoing_decode_tokens":64654,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":595,"cache_hit":0,"new_prefill":23131,"score_linear":64654.0,"score_lmetric":46262},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":75826,"ongoing_decode_tokens":0,"pending_prefill_tokens":75826,"num_requests":1,"active_p_offloads":0,"cached_blocks":778,"cache_hit":0,"new_prefill":23131,"score_linear":75826.0,"score_lmetric":98957},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":811,"cache_hit":0,"new_prefill":23131,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":879,"cache_hit":0,"new_prefill":23131,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":57218,"ongoing_decode_tokens":0,"pending_prefill_tokens":18818,"num_requests":1,"active_p_offloads":0,"cached_blocks":1051,"cache_hit":14848,"new_prefill":8283,"score_linear":42370.0,"score_lmetric":27101},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":74665,"ongoing_decode_tokens":74665,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":540,"cache_hit":0,"new_prefill":23131,"score_linear":74665.0,"score_lmetric":46262},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":801,"cache_hit":0,"new_prefill":23131,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":139890,"ongoing_decode_tokens":139890,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":784,"cache_hit":0,"new_prefill":23131,"score_linear":139890.0,"score_lmetric":46262}],"chosen_score_linear":42370.0,"chosen_score_lmetric":27101,"t_first_token":348553.38996024,"t_first_token_unix":1779779418.4039035,"t_done":348555.564747055,"t_done_unix":1779779420.5786903},{"request_id":"1294611:7:1306822:293","session_id":"1294611","input_length":43378,"t_proxy_recv":348548.335666271,"t_decision_unix":1779779413.3496075,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":30720,"affinity_cache_ratio":0.7081930932730878,"affinity_num_requests":-1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":30720,"estimated_new_tokens":12658,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":64654,"ongoing_decode_tokens":64654,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":595,"cache_hit":0,"new_prefill":43378,"score_linear":64654.0,"score_lmetric":86756},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":75826,"ongoing_decode_tokens":0,"pending_prefill_tokens":75826,"num_requests":1,"active_p_offloads":0,"cached_blocks":778,"cache_hit":0,"new_prefill":43378,"score_linear":75826.0,"score_lmetric":119204},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":811,"cache_hit":0,"new_prefill":43378,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":879,"cache_hit":0,"new_prefill":43378,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1051,"cache_hit":30720,"new_prefill":12658,"score_linear":-30720.0,"score_lmetric":-12658},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":74665,"ongoing_decode_tokens":74665,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":540,"cache_hit":0,"new_prefill":43378,"score_linear":74665.0,"score_lmetric":86756},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":801,"cache_hit":0,"new_prefill":43378,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":253462,"ongoing_decode_tokens":253462,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":543,"cache_hit":12800,"new_prefill":30578,"score_linear":240662.0,"score_lmetric":91734}],"chosen_score_linear":-30720.0,"chosen_score_lmetric":-12658,"t_first_token":348551.567067279,"t_first_token_unix":1779779416.5810106,"t_done":348556.512916775,"t_done_unix":1779779421.52686},{"request_id":"1269373:7:1290314:207","session_id":"1269373","input_length":42745,"t_proxy_recv":348547.100083141,"t_decision_unix":1779779412.1140244,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":38912,"affinity_cache_ratio":0.9103286934144345,"affinity_num_requests":2,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":38912,"estimated_new_tokens":3833,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":64654,"ongoing_decode_tokens":64654,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":595,"cache_hit":7680,"new_prefill":35065,"score_linear":56974.0,"score_lmetric":70130},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":75826,"ongoing_decode_tokens":0,"pending_prefill_tokens":75826,"num_requests":1,"active_p_offloads":0,"cached_blocks":778,"cache_hit":0,"new_prefill":42745,"score_linear":75826.0,"score_lmetric":118571},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":14027,"ongoing_decode_tokens":14027,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":784,"cache_hit":0,"new_prefill":42745,"score_linear":14027.0,"score_lmetric":42745},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":879,"cache_hit":0,"new_prefill":42745,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1051,"cache_hit":0,"new_prefill":42745,"score_linear":0.0,"score_lmetric":-42745},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":74665,"ongoing_decode_tokens":43607,"pending_prefill_tokens":3922,"num_requests":2,"active_p_offloads":0,"cached_blocks":540,"cache_hit":0,"new_prefill":42745,"score_linear":74665.0,"score_lmetric":93334},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":801,"cache_hit":7680,"new_prefill":35065,"score_linear":-7680.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":210717,"ongoing_decode_tokens":210717,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":543,"cache_hit":38912,"new_prefill":3833,"score_linear":171805.0,"score_lmetric":7666}],"chosen_score_linear":171805.0,"chosen_score_lmetric":7666,"t_first_token":348548.216079754,"t_first_token_unix":1779779413.2300234,"t_done":348556.532295166,"t_done_unix":1779779421.5462384},{"request_id":"1303517:3:1309213:303","session_id":"1303517","input_length":23259,"t_proxy_recv":348556.797800347,"t_decision_unix":1779779421.8117416,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":23040,"affinity_cache_ratio":0.9905842899522765,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":23040,"estimated_new_tokens":219,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":34706,"ongoing_decode_tokens":34706,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":598,"cache_hit":0,"new_prefill":23259,"score_linear":34706.0,"score_lmetric":23259},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":75826,"ongoing_decode_tokens":0,"pending_prefill_tokens":75826,"num_requests":1,"active_p_offloads":0,"cached_blocks":778,"cache_hit":0,"new_prefill":23259,"score_linear":75826.0,"score_lmetric":99085},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":811,"cache_hit":0,"new_prefill":23259,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":896,"cache_hit":0,"new_prefill":23259,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":48389,"ongoing_decode_tokens":48389,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1123,"cache_hit":23040,"new_prefill":219,"score_linear":25349.0,"score_lmetric":219},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":548,"cache_hit":0,"new_prefill":23259,"score_linear":43607.0,"score_lmetric":23259},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":801,"cache_hit":0,"new_prefill":23259,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":148491,"ongoing_decode_tokens":97811,"pending_prefill_tokens":7672,"num_requests":2,"active_p_offloads":0,"cached_blocks":794,"cache_hit":0,"new_prefill":23259,"score_linear":148491.0,"score_lmetric":61862}],"chosen_score_linear":25349.0,"chosen_score_lmetric":219,"t_first_token":348556.87278083,"t_first_token_unix":1779779421.8867238,"t_done":348556.977203072,"t_done_unix":1779779421.9911463},{"request_id":"1309040:1:1309040:302","session_id":"1309040","input_length":4679,"t_proxy_recv":348556.171150815,"t_decision_unix":1779779421.1850922,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":4679,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":34706,"ongoing_decode_tokens":34706,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":598,"cache_hit":0,"new_prefill":4679,"score_linear":34706.0,"score_lmetric":4679},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":75826,"ongoing_decode_tokens":0,"pending_prefill_tokens":75826,"num_requests":1,"active_p_offloads":0,"cached_blocks":778,"cache_hit":0,"new_prefill":4679,"score_linear":75826.0,"score_lmetric":80505},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":811,"cache_hit":0,"new_prefill":4679,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":896,"cache_hit":0,"new_prefill":4679,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":43378,"ongoing_decode_tokens":43378,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1098,"cache_hit":0,"new_prefill":4679,"score_linear":43378.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":548,"cache_hit":0,"new_prefill":4679,"score_linear":43607.0,"score_lmetric":4679},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":801,"cache_hit":0,"new_prefill":4679,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":140556,"ongoing_decode_tokens":140556,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":786,"cache_hit":0,"new_prefill":4679,"score_linear":140556.0,"score_lmetric":9358}],"chosen_score_linear":43378.0,"chosen_score_lmetric":0,"t_first_token":348556.481150979,"t_first_token_unix":1779779421.4950943,"t_done":348557.40762598,"t_done_unix":1779779422.421569},{"request_id":"1294611:8:1308970:301","session_id":"1294611","input_length":43710,"t_proxy_recv":348556.52161612,"t_decision_unix":1779779421.5355577,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":43520,"affinity_cache_ratio":0.9956531686113017,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":43520,"estimated_new_tokens":190,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":34706,"ongoing_decode_tokens":34706,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":598,"cache_hit":0,"new_prefill":43710,"score_linear":34706.0,"score_lmetric":43710},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":75826,"ongoing_decode_tokens":0,"pending_prefill_tokens":75826,"num_requests":1,"active_p_offloads":0,"cached_blocks":778,"cache_hit":0,"new_prefill":43710,"score_linear":75826.0,"score_lmetric":119536},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":811,"cache_hit":0,"new_prefill":43710,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":896,"cache_hit":0,"new_prefill":43710,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":4679,"ongoing_decode_tokens":4679,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1123,"cache_hit":43520,"new_prefill":190,"score_linear":-38841.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":548,"cache_hit":0,"new_prefill":43710,"score_linear":43607.0,"score_lmetric":43710},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":801,"cache_hit":0,"new_prefill":43710,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":140556,"ongoing_decode_tokens":140556,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":786,"cache_hit":12800,"new_prefill":30910,"score_linear":127756.0,"score_lmetric":61820}],"chosen_score_linear":-38841.0,"chosen_score_lmetric":0,"t_first_token":348556.673798879,"t_first_token_unix":1779779421.6877422,"t_done":348558.094678166,"t_done_unix":1779779423.1086218},{"request_id":"1301929:1:1301929:265","session_id":"1301929","input_length":34706,"t_proxy_recv":348530.866173256,"t_decision_unix":1779779395.880115,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":3584,"estimated_new_tokens":31122,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":594,"cache_hit":3584,"new_prefill":31122,"score_linear":-3584.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":740,"cache_hit":0,"new_prefill":34706,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":717,"cache_hit":0,"new_prefill":34706,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":867,"cache_hit":0,"new_prefill":34706,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":52835,"ongoing_decode_tokens":23863,"pending_prefill_tokens":18220,"num_requests":1,"active_p_offloads":0,"cached_blocks":927,"cache_hit":0,"new_prefill":34706,"score_linear":52835.0,"score_lmetric":52926},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":69555,"ongoing_decode_tokens":0,"pending_prefill_tokens":30643,"num_requests":2,"active_p_offloads":0,"cached_blocks":538,"cache_hit":0,"new_prefill":34706,"score_linear":69555.0,"score_lmetric":130698},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":43181,"ongoing_decode_tokens":43181,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":762,"cache_hit":0,"new_prefill":34706,"score_linear":43181.0,"score_lmetric":34706},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":241141,"ongoing_decode_tokens":153400,"pending_prefill_tokens":7357,"num_requests":3,"active_p_offloads":0,"cached_blocks":512,"cache_hit":0,"new_prefill":34706,"score_linear":241141.0,"score_lmetric":126189}],"chosen_score_linear":-3584.0,"chosen_score_lmetric":0,"t_first_token":348535.67020197,"t_first_token_unix":1779779400.6841455,"t_done":348559.386379596,"t_done_unix":1779779424.4003232},{"request_id":"1266668:11:1300918:260","session_id":"1266668","input_length":97811,"t_proxy_recv":348554.186008193,"t_decision_unix":1779779419.1999497,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":97280,"affinity_cache_ratio":0.9945711627526557,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":97280,"estimated_new_tokens":531,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":34706,"ongoing_decode_tokens":34706,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":598,"cache_hit":7680,"new_prefill":90131,"score_linear":27026.0,"score_lmetric":90131},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":75826,"ongoing_decode_tokens":0,"pending_prefill_tokens":75826,"num_requests":1,"active_p_offloads":0,"cached_blocks":778,"cache_hit":0,"new_prefill":97811,"score_linear":75826.0,"score_lmetric":173637},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":811,"cache_hit":0,"new_prefill":97811,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":896,"cache_hit":0,"new_prefill":97811,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":103245,"ongoing_decode_tokens":80349,"pending_prefill_tokens":10096,"num_requests":3,"active_p_offloads":0,"cached_blocks":1051,"cache_hit":0,"new_prefill":97811,"score_linear":103245.0,"score_lmetric":323721},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":548,"cache_hit":0,"new_prefill":97811,"score_linear":43607.0,"score_lmetric":97811},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":801,"cache_hit":7680,"new_prefill":90131,"score_linear":-7680.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":42745,"ongoing_decode_tokens":42745,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":786,"cache_hit":97280,"new_prefill":531,"score_linear":-54535.0,"score_lmetric":531}],"chosen_score_linear":-54535.0,"chosen_score_lmetric":531,"t_first_token":348554.673306282,"t_first_token_unix":1779779419.6872504,"t_done":348561.061734613,"t_done_unix":1779779426.075678},{"request_id":"1309252:1:1309252:304","session_id":"1309252","input_length":24545,"t_proxy_recv":348557.002812121,"t_decision_unix":1779779422.0167537,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":24545,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":34706,"ongoing_decode_tokens":34706,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":598,"cache_hit":0,"new_prefill":24545,"score_linear":34706.0,"score_lmetric":24545},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":75826,"ongoing_decode_tokens":0,"pending_prefill_tokens":75826,"num_requests":1,"active_p_offloads":0,"cached_blocks":778,"cache_hit":0,"new_prefill":24545,"score_linear":75826.0,"score_lmetric":100371},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":811,"cache_hit":0,"new_prefill":24545,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":896,"cache_hit":0,"new_prefill":24545,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":48389,"ongoing_decode_tokens":48389,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1123,"cache_hit":0,"new_prefill":24545,"score_linear":48389.0,"score_lmetric":24545},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":548,"cache_hit":0,"new_prefill":24545,"score_linear":43607.0,"score_lmetric":24545},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":801,"cache_hit":0,"new_prefill":24545,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":148491,"ongoing_decode_tokens":97811,"pending_prefill_tokens":7672,"num_requests":2,"active_p_offloads":0,"cached_blocks":794,"cache_hit":0,"new_prefill":24545,"score_linear":148491.0,"score_lmetric":64434}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348559.82420286,"t_first_token_unix":1779779424.8381472,"t_done":348561.706330499,"t_done_unix":1779779426.7202735},{"request_id":"1279412:11:1310014:305","session_id":"1279412","input_length":64558,"t_proxy_recv":348559.638425041,"t_decision_unix":1779779424.6523666,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":61440,"affinity_cache_ratio":0.9517023451779795,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":61440,"estimated_new_tokens":3118,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":660,"cache_hit":61440,"new_prefill":3118,"score_linear":-61440.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":75826,"ongoing_decode_tokens":0,"pending_prefill_tokens":75826,"num_requests":1,"active_p_offloads":0,"cached_blocks":778,"cache_hit":0,"new_prefill":64558,"score_linear":75826.0,"score_lmetric":140384},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":24545,"ongoing_decode_tokens":0,"pending_prefill_tokens":24545,"num_requests":1,"active_p_offloads":0,"cached_blocks":811,"cache_hit":0,"new_prefill":64558,"score_linear":24545.0,"score_lmetric":89103},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":896,"cache_hit":0,"new_prefill":64558,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1132,"cache_hit":0,"new_prefill":64558,"score_linear":0.0,"score_lmetric":-64558},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":548,"cache_hit":0,"new_prefill":64558,"score_linear":43607.0,"score_lmetric":64558},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":801,"cache_hit":12800,"new_prefill":51758,"score_linear":-12800.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":148491,"ongoing_decode_tokens":148491,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":794,"cache_hit":12800,"new_prefill":51758,"score_linear":135691.0,"score_lmetric":103516}],"chosen_score_linear":-61440.0,"chosen_score_lmetric":0,"t_first_token":348560.941359448,"t_first_token_unix":1779779425.9553032,"t_done":348562.334014587,"t_done_unix":1779779427.347958},{"request_id":"1305420:1:1305420:285","session_id":"1305420","input_length":75826,"t_proxy_recv":348543.324755796,"t_decision_unix":1779779408.338697,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":75826,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":64654,"ongoing_decode_tokens":64654,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":594,"cache_hit":0,"new_prefill":75826,"score_linear":64654.0,"score_lmetric":151652},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":778,"cache_hit":0,"new_prefill":75826,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":784,"cache_hit":0,"new_prefill":75826,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":879,"cache_hit":0,"new_prefill":75826,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":30995,"ongoing_decode_tokens":30995,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1025,"cache_hit":0,"new_prefill":75826,"score_linear":30995.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":540,"cache_hit":0,"new_prefill":75826,"score_linear":43607.0,"score_lmetric":75826},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":794,"cache_hit":0,"new_prefill":75826,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":249505,"ongoing_decode_tokens":153400,"pending_prefill_tokens":8041,"num_requests":3,"active_p_offloads":0,"cached_blocks":527,"cache_hit":0,"new_prefill":75826,"score_linear":249505.0,"score_lmetric":251601}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348563.02171563,"t_first_token_unix":1779779428.0356598,"t_done":348563.654284863,"t_done_unix":1779779428.6682289},{"request_id":"1304943:2:1311189:307","session_id":"1304943","input_length":13164,"t_proxy_recv":348563.716435336,"t_decision_unix":1779779428.7303767,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":2560,"affinity_cache_ratio":0.19446976602856275,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":2560,"estimated_new_tokens":10604,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":666,"cache_hit":0,"new_prefill":13164,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":926,"cache_hit":0,"new_prefill":13164,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":859,"cache_hit":0,"new_prefill":13164,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":896,"cache_hit":0,"new_prefill":13164,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":18089,"ongoing_decode_tokens":18089,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1132,"cache_hit":2560,"new_prefill":10604,"score_linear":15529.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":548,"cache_hit":0,"new_prefill":13164,"score_linear":43607.0,"score_lmetric":13164},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":801,"cache_hit":0,"new_prefill":13164,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":172648,"ongoing_decode_tokens":50680,"pending_prefill_tokens":24176,"num_requests":2,"active_p_offloads":0,"cached_blocks":795,"cache_hit":0,"new_prefill":13164,"score_linear":172648.0,"score_lmetric":74680}],"chosen_score_linear":15529.0,"chosen_score_lmetric":0,"t_first_token":348564.701238356,"t_first_token_unix":1779779429.7151825,"t_done":348565.072926609,"t_done_unix":1779779430.0868702},{"request_id":"1310590:1:1310590:306","session_id":"1310590","input_length":18089,"t_proxy_recv":348561.585487258,"t_decision_unix":1779779426.5994284,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":-18089,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":18089,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":64558,"ongoing_decode_tokens":64558,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":660,"cache_hit":0,"new_prefill":18089,"score_linear":64558.0,"score_lmetric":18089},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":75826,"ongoing_decode_tokens":0,"pending_prefill_tokens":75826,"num_requests":1,"active_p_offloads":0,"cached_blocks":778,"cache_hit":0,"new_prefill":18089,"score_linear":75826.0,"score_lmetric":93915},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":24545,"ongoing_decode_tokens":24545,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":811,"cache_hit":0,"new_prefill":18089,"score_linear":24545.0,"score_lmetric":18089},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":896,"cache_hit":0,"new_prefill":18089,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1132,"cache_hit":0,"new_prefill":18089,"score_linear":0.0,"score_lmetric":-18089},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":548,"cache_hit":0,"new_prefill":18089,"score_linear":43607.0,"score_lmetric":18089},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":801,"cache_hit":0,"new_prefill":18089,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":172648,"ongoing_decode_tokens":50680,"pending_prefill_tokens":24176,"num_requests":2,"active_p_offloads":0,"cached_blocks":795,"cache_hit":0,"new_prefill":18089,"score_linear":172648.0,"score_lmetric":84530}],"chosen_score_linear":0.0,"chosen_score_lmetric":-18089,"t_first_token":348563.380767119,"t_first_token_unix":1779779428.3947108,"t_done":348565.154485565,"t_done_unix":1779779430.168429},{"request_id":"1303517:4:1311293:308","session_id":"1303517","input_length":24638,"t_proxy_recv":348564.03348019,"t_decision_unix":1779779429.047421,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":23040,"affinity_cache_ratio":0.9351408393538436,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":23040,"estimated_new_tokens":1598,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":666,"cache_hit":0,"new_prefill":24638,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":926,"cache_hit":0,"new_prefill":24638,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":859,"cache_hit":0,"new_prefill":24638,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":896,"cache_hit":0,"new_prefill":24638,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":31253,"ongoing_decode_tokens":18089,"pending_prefill_tokens":10604,"num_requests":1,"active_p_offloads":0,"cached_blocks":1132,"cache_hit":23040,"new_prefill":1598,"score_linear":8213.0,"score_lmetric":12202},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":548,"cache_hit":0,"new_prefill":24638,"score_linear":43607.0,"score_lmetric":24638},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":801,"cache_hit":0,"new_prefill":24638,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":172648,"ongoing_decode_tokens":50680,"pending_prefill_tokens":24176,"num_requests":2,"active_p_offloads":0,"cached_blocks":795,"cache_hit":0,"new_prefill":24638,"score_linear":172648.0,"score_lmetric":97628}],"chosen_score_linear":8213.0,"chosen_score_lmetric":12202,"t_first_token":348565.002142198,"t_first_token_unix":1779779430.0160854,"t_done":348565.217019768,"t_done_unix":1779779430.230963},{"request_id":"1277428:3:1311644:311","session_id":"1277428","input_length":31611,"t_proxy_recv":348565.340535528,"t_decision_unix":1779779430.3544772,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":31232,"affinity_cache_ratio":0.9880105026731201,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":31232,"estimated_new_tokens":379,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":666,"cache_hit":0,"new_prefill":31611,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":926,"cache_hit":4608,"new_prefill":27003,"score_linear":-4608.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":44792,"ongoing_decode_tokens":0,"pending_prefill_tokens":20216,"num_requests":1,"active_p_offloads":0,"cached_blocks":859,"cache_hit":0,"new_prefill":31611,"score_linear":44792.0,"score_lmetric":51827},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":8996,"ongoing_decode_tokens":0,"pending_prefill_tokens":292,"num_requests":1,"active_p_offloads":0,"cached_blocks":896,"cache_hit":17408,"new_prefill":14203,"score_linear":-8412.0,"score_lmetric":14495},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1190,"cache_hit":0,"new_prefill":31611,"score_linear":0.0,"score_lmetric":-31611},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":548,"cache_hit":31232,"new_prefill":379,"score_linear":12375.0,"score_lmetric":379},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":801,"cache_hit":0,"new_prefill":31611,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":172648,"ongoing_decode_tokens":50680,"pending_prefill_tokens":24176,"num_requests":2,"active_p_offloads":0,"cached_blocks":795,"cache_hit":0,"new_prefill":31611,"score_linear":172648.0,"score_lmetric":111574}],"chosen_score_linear":12375.0,"chosen_score_lmetric":379,"t_first_token":348565.412018361,"t_first_token_unix":1779779430.425962,"t_done":348565.93943384,"t_done_unix":1779779430.9533775},{"request_id":"1297811:2:1300578:258","session_id":"1297811","input_length":43607,"t_proxy_recv":348526.201406164,"t_decision_unix":1779779391.2153475,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":13824,"affinity_cache_ratio":0.3170133235489715,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":13824,"estimated_new_tokens":29783,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":60678,"ongoing_decode_tokens":60678,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":594,"cache_hit":0,"new_prefill":43607,"score_linear":60678.0,"score_lmetric":43607},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":740,"cache_hit":0,"new_prefill":43607,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":62414,"ongoing_decode_tokens":0,"pending_prefill_tokens":43470,"num_requests":1,"active_p_offloads":0,"cached_blocks":633,"cache_hit":0,"new_prefill":43607,"score_linear":62414.0,"score_lmetric":87077},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":867,"cache_hit":0,"new_prefill":43607,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":83370,"ongoing_decode_tokens":0,"pending_prefill_tokens":59818,"num_requests":1,"active_p_offloads":0,"cached_blocks":811,"cache_hit":0,"new_prefill":43607,"score_linear":83370.0,"score_lmetric":103425},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":538,"cache_hit":13824,"new_prefill":29783,"score_linear":-13824.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":18893,"ongoing_decode_tokens":18893,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":747,"cache_hit":0,"new_prefill":43607,"score_linear":18893.0,"score_lmetric":43607},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":225261,"ongoing_decode_tokens":225261,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":468,"cache_hit":0,"new_prefill":43607,"score_linear":225261.0,"score_lmetric":130821}],"chosen_score_linear":-13824.0,"chosen_score_lmetric":0,"t_first_token":348532.481059165,"t_first_token_unix":1779779397.4950025,"t_done":348566.978018125,"t_done_unix":1779779431.9919617},{"request_id":"1258908:2:1311606:310","session_id":"1258908","input_length":8996,"t_proxy_recv":348565.160933442,"t_decision_unix":1779779430.174875,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":8704,"affinity_cache_ratio":0.9675411293908404,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":8704,"estimated_new_tokens":292,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":666,"cache_hit":0,"new_prefill":8996,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":926,"cache_hit":0,"new_prefill":8996,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":44792,"ongoing_decode_tokens":0,"pending_prefill_tokens":20216,"num_requests":1,"active_p_offloads":0,"cached_blocks":859,"cache_hit":0,"new_prefill":8996,"score_linear":44792.0,"score_lmetric":29212},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":896,"cache_hit":8704,"new_prefill":292,"score_linear":-8704.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":24638,"ongoing_decode_tokens":24638,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1187,"cache_hit":0,"new_prefill":8996,"score_linear":24638.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":548,"cache_hit":0,"new_prefill":8996,"score_linear":43607.0,"score_lmetric":8996},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":801,"cache_hit":0,"new_prefill":8996,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":172648,"ongoing_decode_tokens":50680,"pending_prefill_tokens":24176,"num_requests":2,"active_p_offloads":0,"cached_blocks":795,"cache_hit":0,"new_prefill":8996,"score_linear":172648.0,"score_lmetric":66344}],"chosen_score_linear":-8704.0,"chosen_score_lmetric":0,"t_first_token":348565.831051785,"t_first_token_unix":1779779430.844995,"t_done":348567.745154778,"t_done_unix":1779779432.7590983},{"request_id":"1311753:1:1311753:312","session_id":"1311753","input_length":20175,"t_proxy_recv":348565.733412432,"t_decision_unix":1779779430.7473536,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":-20175,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":20175,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":666,"cache_hit":0,"new_prefill":20175,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":926,"cache_hit":0,"new_prefill":20175,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":44792,"ongoing_decode_tokens":0,"pending_prefill_tokens":20216,"num_requests":1,"active_p_offloads":0,"cached_blocks":859,"cache_hit":0,"new_prefill":20175,"score_linear":44792.0,"score_lmetric":40391},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":8996,"ongoing_decode_tokens":0,"pending_prefill_tokens":292,"num_requests":1,"active_p_offloads":0,"cached_blocks":896,"cache_hit":0,"new_prefill":20175,"score_linear":8996.0,"score_lmetric":20467},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1190,"cache_hit":0,"new_prefill":20175,"score_linear":0.0,"score_lmetric":-20175},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":75218,"ongoing_decode_tokens":75218,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":548,"cache_hit":0,"new_prefill":20175,"score_linear":75218.0,"score_lmetric":40350},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":801,"cache_hit":0,"new_prefill":20175,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":172648,"ongoing_decode_tokens":50680,"pending_prefill_tokens":24176,"num_requests":2,"active_p_offloads":0,"cached_blocks":795,"cache_hit":0,"new_prefill":20175,"score_linear":172648.0,"score_lmetric":88702}],"chosen_score_linear":0.0,"chosen_score_lmetric":-20175,"t_first_token":348567.84446458,"t_first_token_unix":1779779432.858408,"t_done":348568.027053644,"t_done_unix":1779779433.0409966},{"request_id":"1303517:5:1313040:317","session_id":"1303517","input_length":25637,"t_proxy_recv":348570.080574908,"t_decision_unix":1779779435.0945165,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":24576,"affinity_cache_ratio":0.9586145024768888,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":24576,"estimated_new_tokens":1061,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":666,"cache_hit":0,"new_prefill":25637,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":26885,"ongoing_decode_tokens":26885,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":926,"cache_hit":0,"new_prefill":25637,"score_linear":26885.0,"score_lmetric":25637},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":44792,"ongoing_decode_tokens":44792,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":859,"cache_hit":0,"new_prefill":25637,"score_linear":44792.0,"score_lmetric":25637},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":897,"cache_hit":0,"new_prefill":25637,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":57695,"ongoing_decode_tokens":57695,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1229,"cache_hit":24576,"new_prefill":1061,"score_linear":33119.0,"score_lmetric":1061},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":608,"cache_hit":0,"new_prefill":25637,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":801,"cache_hit":0,"new_prefill":25637,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":172648,"ongoing_decode_tokens":50680,"pending_prefill_tokens":24176,"num_requests":2,"active_p_offloads":0,"cached_blocks":795,"cache_hit":0,"new_prefill":25637,"score_linear":172648.0,"score_lmetric":99626}],"chosen_score_linear":33119.0,"chosen_score_lmetric":1061,"t_first_token":348570.358031258,"t_first_token_unix":1779779435.3719745,"t_done":348570.428887641,"t_done_unix":1779779435.4428306},{"request_id":"1309252:2:1311531:309","session_id":"1309252","input_length":44792,"t_proxy_recv":348564.932424556,"t_decision_unix":1779779429.9463637,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":24576,"affinity_cache_ratio":0.5486694052509377,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":24576,"estimated_new_tokens":20216,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":666,"cache_hit":0,"new_prefill":44792,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":926,"cache_hit":0,"new_prefill":44792,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":859,"cache_hit":24576,"new_prefill":20216,"score_linear":-24576.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":896,"cache_hit":0,"new_prefill":44792,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":55891,"ongoing_decode_tokens":31253,"pending_prefill_tokens":1598,"num_requests":2,"active_p_offloads":0,"cached_blocks":1132,"cache_hit":0,"new_prefill":44792,"score_linear":55891.0,"score_lmetric":92780},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":548,"cache_hit":0,"new_prefill":44792,"score_linear":43607.0,"score_lmetric":44792},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":801,"cache_hit":0,"new_prefill":44792,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":172648,"ongoing_decode_tokens":50680,"pending_prefill_tokens":24176,"num_requests":2,"active_p_offloads":0,"cached_blocks":795,"cache_hit":0,"new_prefill":44792,"score_linear":172648.0,"score_lmetric":137936}],"chosen_score_linear":-24576.0,"chosen_score_lmetric":0,"t_first_token":348569.855471237,"t_first_token_unix":1779779434.8694143,"t_done":348570.538672588,"t_done_unix":1779779435.5526156},{"request_id":"1294611:9:1312398:313","session_id":"1294611","input_length":44335,"t_proxy_recv":348568.016424051,"t_decision_unix":1779779433.0303655,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":43520,"affinity_cache_ratio":0.9816172324348709,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":43520,"estimated_new_tokens":815,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":666,"cache_hit":0,"new_prefill":44335,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":926,"cache_hit":0,"new_prefill":44335,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":44792,"ongoing_decode_tokens":0,"pending_prefill_tokens":20216,"num_requests":1,"active_p_offloads":0,"cached_blocks":859,"cache_hit":0,"new_prefill":44335,"score_linear":44792.0,"score_lmetric":64551},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":897,"cache_hit":0,"new_prefill":44335,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":20175,"ongoing_decode_tokens":20175,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1190,"cache_hit":43520,"new_prefill":815,"score_linear":-23345.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":608,"cache_hit":0,"new_prefill":44335,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":801,"cache_hit":0,"new_prefill":44335,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":172648,"ongoing_decode_tokens":50680,"pending_prefill_tokens":24176,"num_requests":2,"active_p_offloads":0,"cached_blocks":795,"cache_hit":12800,"new_prefill":31535,"score_linear":159848.0,"score_lmetric":111422}],"chosen_score_linear":-23345.0,"chosen_score_lmetric":0,"t_first_token":348568.260054868,"t_first_token_unix":1779779433.273998,"t_done":348570.97827047,"t_done_unix":1779779435.9922135},{"request_id":"1306347:2:1312642:314","session_id":"1306347","input_length":13360,"t_proxy_recv":348568.907721237,"t_decision_unix":1779779433.9216626,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":512,"affinity_cache_ratio":0.03832335329341317,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":512,"estimated_new_tokens":12848,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":666,"cache_hit":0,"new_prefill":13360,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":926,"cache_hit":0,"new_prefill":13360,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":44792,"ongoing_decode_tokens":0,"pending_prefill_tokens":20216,"num_requests":1,"active_p_offloads":0,"cached_blocks":859,"cache_hit":0,"new_prefill":13360,"score_linear":44792.0,"score_lmetric":33576},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":897,"cache_hit":0,"new_prefill":13360,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":44335,"ongoing_decode_tokens":44335,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1229,"cache_hit":512,"new_prefill":12848,"score_linear":43823.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":608,"cache_hit":0,"new_prefill":13360,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":801,"cache_hit":0,"new_prefill":13360,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":172648,"ongoing_decode_tokens":50680,"pending_prefill_tokens":24176,"num_requests":2,"active_p_offloads":0,"cached_blocks":795,"cache_hit":0,"new_prefill":13360,"score_linear":172648.0,"score_lmetric":75072}],"chosen_score_linear":43823.0,"chosen_score_lmetric":0,"t_first_token":348570.013256097,"t_first_token_unix":1779779435.0271993,"t_done":348571.061872394,"t_done_unix":1779779436.0758154},{"request_id":"1268861:4:1312689:315","session_id":"1268861","input_length":26885,"t_proxy_recv":348569.076426108,"t_decision_unix":1779779434.0903673,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":21504,"affinity_cache_ratio":0.7998512181513855,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":21504,"estimated_new_tokens":5381,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":666,"cache_hit":0,"new_prefill":26885,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":926,"cache_hit":21504,"new_prefill":5381,"score_linear":-21504.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":44792,"ongoing_decode_tokens":0,"pending_prefill_tokens":20216,"num_requests":1,"active_p_offloads":0,"cached_blocks":859,"cache_hit":0,"new_prefill":26885,"score_linear":44792.0,"score_lmetric":47101},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":897,"cache_hit":0,"new_prefill":26885,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":57695,"ongoing_decode_tokens":44335,"pending_prefill_tokens":12848,"num_requests":1,"active_p_offloads":0,"cached_blocks":1229,"cache_hit":0,"new_prefill":26885,"score_linear":57695.0,"score_lmetric":39733},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":608,"cache_hit":0,"new_prefill":26885,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":801,"cache_hit":0,"new_prefill":26885,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":172648,"ongoing_decode_tokens":50680,"pending_prefill_tokens":24176,"num_requests":2,"active_p_offloads":0,"cached_blocks":795,"cache_hit":12288,"new_prefill":14597,"score_linear":160360.0,"score_lmetric":77546}],"chosen_score_linear":-21504.0,"chosen_score_lmetric":0,"t_first_token":348570.01064388,"t_first_token_unix":1779779435.0245876,"t_done":348571.269536425,"t_done_unix":1779779436.2834795},{"request_id":"1313181:1:1313181:318","session_id":"1313181","input_length":22686,"t_proxy_recv":348570.695452488,"t_decision_unix":1779779435.7093942,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":12800,"estimated_new_tokens":9886,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":666,"cache_hit":12800,"new_prefill":9886,"score_linear":-12800.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":26885,"ongoing_decode_tokens":26885,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":926,"cache_hit":0,"new_prefill":22686,"score_linear":26885.0,"score_lmetric":22686},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":898,"cache_hit":0,"new_prefill":22686,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":897,"cache_hit":0,"new_prefill":22686,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":57695,"ongoing_decode_tokens":57695,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1231,"cache_hit":0,"new_prefill":22686,"score_linear":57695.0,"score_lmetric":22686},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":608,"cache_hit":0,"new_prefill":22686,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":801,"cache_hit":12800,"new_prefill":9886,"score_linear":-12800.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":172648,"ongoing_decode_tokens":50680,"pending_prefill_tokens":24176,"num_requests":2,"active_p_offloads":0,"cached_blocks":795,"cache_hit":12800,"new_prefill":9886,"score_linear":159848.0,"score_lmetric":68124}],"chosen_score_linear":-12800.0,"chosen_score_lmetric":0,"t_first_token":348572.189781358,"t_first_token_unix":1779779437.2037249,"t_done":348573.359079393,"t_done_unix":1779779438.3730226},{"request_id":"1279412:12:1313518:319","session_id":"1279412","input_length":64755,"t_proxy_recv":348571.818565715,"t_decision_unix":1779779436.8325071,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":64512,"affinity_cache_ratio":0.9962473940236275,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":64512,"estimated_new_tokens":243,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":22686,"ongoing_decode_tokens":0,"pending_prefill_tokens":9886,"num_requests":1,"active_p_offloads":0,"cached_blocks":666,"cache_hit":64512,"new_prefill":243,"score_linear":-41826.0,"score_lmetric":10129},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":936,"cache_hit":0,"new_prefill":64755,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":898,"cache_hit":0,"new_prefill":64755,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":897,"cache_hit":0,"new_prefill":64755,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1257,"cache_hit":0,"new_prefill":64755,"score_linear":0.0,"score_lmetric":-64755},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":608,"cache_hit":0,"new_prefill":64755,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":801,"cache_hit":12800,"new_prefill":51955,"score_linear":-12800.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":172648,"ongoing_decode_tokens":50680,"pending_prefill_tokens":24176,"num_requests":2,"active_p_offloads":0,"cached_blocks":795,"cache_hit":12800,"new_prefill":51955,"score_linear":159848.0,"score_lmetric":152262}],"chosen_score_linear":-41826.0,"chosen_score_lmetric":10129,"t_first_token":348572.264148838,"t_first_token_unix":1779779437.2780924,"t_done":348573.784539632,"t_done_unix":1779779438.798483},{"request_id":"1313181:2:1314940:323","session_id":"1313181","input_length":24306,"t_proxy_recv":348576.888937946,"t_decision_unix":1779779441.9028795,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":22528,"affinity_cache_ratio":0.9268493376121122,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":22528,"estimated_new_tokens":1778,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":685,"cache_hit":22528,"new_prefill":1778,"score_linear":-22528.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":936,"cache_hit":0,"new_prefill":24306,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":898,"cache_hit":0,"new_prefill":24306,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":897,"cache_hit":0,"new_prefill":24306,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":102413,"ongoing_decode_tokens":0,"pending_prefill_tokens":58381,"num_requests":1,"active_p_offloads":0,"cached_blocks":1257,"cache_hit":0,"new_prefill":24306,"score_linear":102413.0,"score_lmetric":82687},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":608,"cache_hit":0,"new_prefill":24306,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":801,"cache_hit":12800,"new_prefill":11506,"score_linear":-12800.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":172648,"ongoing_decode_tokens":50680,"pending_prefill_tokens":24176,"num_requests":2,"active_p_offloads":0,"cached_blocks":795,"cache_hit":12800,"new_prefill":11506,"score_linear":159848.0,"score_lmetric":71364}],"chosen_score_linear":-22528.0,"chosen_score_lmetric":0,"t_first_token":348577.218142709,"t_first_token_unix":1779779442.2320864,"t_done":348577.819769065,"t_done_unix":1779779442.833712},{"request_id":"1315313:1:1315313:325","session_id":"1315313","input_length":7,"t_proxy_recv":348578.117032929,"t_decision_unix":1779779443.1309736,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":7,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":688,"cache_hit":0,"new_prefill":7,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":936,"cache_hit":0,"new_prefill":7,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":898,"cache_hit":0,"new_prefill":7,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":897,"cache_hit":0,"new_prefill":7,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":134853,"ongoing_decode_tokens":0,"pending_prefill_tokens":72901,"num_requests":2,"active_p_offloads":0,"cached_blocks":1257,"cache_hit":0,"new_prefill":7,"score_linear":134853.0,"score_lmetric":145816},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":608,"cache_hit":0,"new_prefill":7,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":801,"cache_hit":0,"new_prefill":7,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":172648,"ongoing_decode_tokens":172648,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":795,"cache_hit":0,"new_prefill":7,"score_linear":172648.0,"score_lmetric":14}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348578.134425839,"t_first_token_unix":1779779443.1483688,"t_done":348578.138677542,"t_done_unix":1779779443.1526203},{"request_id":"1269373:8:1295332:230","session_id":"1269373","input_length":50680,"t_proxy_recv":348556.543277808,"t_decision_unix":1779779421.5572193,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":43008,"affinity_cache_ratio":0.8486187845303867,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":43008,"estimated_new_tokens":7672,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":34706,"ongoing_decode_tokens":34706,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":598,"cache_hit":7680,"new_prefill":43000,"score_linear":27026.0,"score_lmetric":43000},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":75826,"ongoing_decode_tokens":0,"pending_prefill_tokens":75826,"num_requests":1,"active_p_offloads":0,"cached_blocks":778,"cache_hit":0,"new_prefill":50680,"score_linear":75826.0,"score_lmetric":126506},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":811,"cache_hit":0,"new_prefill":50680,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":896,"cache_hit":0,"new_prefill":50680,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":48389,"ongoing_decode_tokens":4679,"pending_prefill_tokens":190,"num_requests":1,"active_p_offloads":0,"cached_blocks":1123,"cache_hit":0,"new_prefill":50680,"score_linear":48389.0,"score_lmetric":50870},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":548,"cache_hit":0,"new_prefill":50680,"score_linear":43607.0,"score_lmetric":50680},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":801,"cache_hit":7680,"new_prefill":43000,"score_linear":-7680.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":97811,"ongoing_decode_tokens":97811,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":794,"cache_hit":43008,"new_prefill":7672,"score_linear":54803.0,"score_lmetric":7672}],"chosen_score_linear":54803.0,"chosen_score_lmetric":7672,"t_first_token":348558.86395697,"t_first_token_unix":1779779423.8779004,"t_done":348582.041484354,"t_done_unix":1779779447.0554278},{"request_id":"1309040:2:1315488:326","session_id":"1309040","input_length":23097,"t_proxy_recv":348578.837366563,"t_decision_unix":1779779443.8513076,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":4,"chosen_idx":0,"affinity_cache_hit":4608,"affinity_cache_ratio":0.1995064294064164,"affinity_num_requests":2,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":23097,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":688,"cache_hit":0,"new_prefill":23097,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":936,"cache_hit":0,"new_prefill":23097,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":898,"cache_hit":0,"new_prefill":23097,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":897,"cache_hit":0,"new_prefill":23097,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":134853,"ongoing_decode_tokens":0,"pending_prefill_tokens":72901,"num_requests":2,"active_p_offloads":0,"cached_blocks":1257,"cache_hit":4608,"new_prefill":18489,"score_linear":130245.0,"score_lmetric":182780},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":608,"cache_hit":0,"new_prefill":23097,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":801,"cache_hit":0,"new_prefill":23097,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":172648,"ongoing_decode_tokens":172648,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":795,"cache_hit":0,"new_prefill":23097,"score_linear":172648.0,"score_lmetric":46194}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348581.40050025,"t_first_token_unix":1779779446.4144435,"t_done":348582.067911748,"t_done_unix":1779779447.0818546},{"request_id":"1315567:1:1315567:327","session_id":"1315567","input_length":16960,"t_proxy_recv":348579.112958493,"t_decision_unix":1779779444.1268995,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":7680,"estimated_new_tokens":9280,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":23097,"ongoing_decode_tokens":0,"pending_prefill_tokens":23097,"num_requests":1,"active_p_offloads":0,"cached_blocks":688,"cache_hit":7680,"new_prefill":9280,"score_linear":15417.0,"score_lmetric":32377},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":936,"cache_hit":0,"new_prefill":16960,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":898,"cache_hit":0,"new_prefill":16960,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":897,"cache_hit":0,"new_prefill":16960,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":134853,"ongoing_decode_tokens":0,"pending_prefill_tokens":72901,"num_requests":2,"active_p_offloads":0,"cached_blocks":1257,"cache_hit":0,"new_prefill":16960,"score_linear":134853.0,"score_lmetric":179722},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":608,"cache_hit":0,"new_prefill":16960,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":801,"cache_hit":7680,"new_prefill":9280,"score_linear":-7680.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":172648,"ongoing_decode_tokens":172648,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":795,"cache_hit":7680,"new_prefill":9280,"score_linear":164968.0,"score_lmetric":18560}],"chosen_score_linear":-7680.0,"chosen_score_lmetric":0,"t_first_token":348580.232162201,"t_first_token_unix":1779779445.2461064,"t_done":348583.341183865,"t_done_unix":1779779448.3551273},{"request_id":"1316848:1:1316848:331","session_id":"1316848","input_length":5985,"t_proxy_recv":348583.458832454,"t_decision_unix":1779779448.4727736,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":5985,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":64949,"ongoing_decode_tokens":64949,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":733,"cache_hit":0,"new_prefill":5985,"score_linear":64949.0,"score_lmetric":5985},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":936,"cache_hit":0,"new_prefill":5985,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":898,"cache_hit":0,"new_prefill":5985,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":897,"cache_hit":0,"new_prefill":5985,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":134853,"ongoing_decode_tokens":0,"pending_prefill_tokens":72901,"num_requests":2,"active_p_offloads":0,"cached_blocks":1257,"cache_hit":1024,"new_prefill":4961,"score_linear":133829.0,"score_lmetric":155724},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":24487,"ongoing_decode_tokens":0,"pending_prefill_tokens":24487,"num_requests":1,"active_p_offloads":0,"cached_blocks":608,"cache_hit":0,"new_prefill":5985,"score_linear":24487.0,"score_lmetric":30472},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":5985,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":176419,"ongoing_decode_tokens":176419,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":810,"cache_hit":0,"new_prefill":5985,"score_linear":176419.0,"score_lmetric":11970}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348583.851778979,"t_first_token_unix":1779779448.8657224,"t_done":348584.03828467,"t_done_unix":1779779449.0522287},{"request_id":"1279412:13:1315780:328","session_id":"1279412","input_length":64949,"t_proxy_recv":348579.907825964,"t_decision_unix":1779779444.9217672,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":64512,"affinity_cache_ratio":0.9932716439052179,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":64512,"estimated_new_tokens":437,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":23097,"ongoing_decode_tokens":0,"pending_prefill_tokens":23097,"num_requests":1,"active_p_offloads":0,"cached_blocks":688,"cache_hit":64512,"new_prefill":437,"score_linear":-41415.0,"score_lmetric":23534},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":936,"cache_hit":0,"new_prefill":64949,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":898,"cache_hit":0,"new_prefill":64949,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":897,"cache_hit":0,"new_prefill":64949,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":134853,"ongoing_decode_tokens":0,"pending_prefill_tokens":72901,"num_requests":2,"active_p_offloads":0,"cached_blocks":1257,"cache_hit":0,"new_prefill":64949,"score_linear":134853.0,"score_lmetric":275700},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":608,"cache_hit":0,"new_prefill":64949,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":16960,"ongoing_decode_tokens":0,"pending_prefill_tokens":9280,"num_requests":1,"active_p_offloads":0,"cached_blocks":801,"cache_hit":12800,"new_prefill":52149,"score_linear":4160.0,"score_lmetric":61429},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":172648,"ongoing_decode_tokens":172648,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":795,"cache_hit":12800,"new_prefill":52149,"score_linear":159848.0,"score_lmetric":104298}],"chosen_score_linear":-41415.0,"chosen_score_lmetric":23534,"t_first_token":348581.478284098,"t_first_token_unix":1779779446.4922278,"t_done":348584.216815632,"t_done_unix":1779779449.2307594},{"request_id":"1316201:1:1316201:329","session_id":"1316201","input_length":24487,"t_proxy_recv":348581.394598947,"t_decision_unix":1779779446.40854,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":24487,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":88046,"ongoing_decode_tokens":0,"pending_prefill_tokens":23534,"num_requests":2,"active_p_offloads":0,"cached_blocks":688,"cache_hit":0,"new_prefill":24487,"score_linear":88046.0,"score_lmetric":96042},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":936,"cache_hit":0,"new_prefill":24487,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":898,"cache_hit":0,"new_prefill":24487,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":897,"cache_hit":0,"new_prefill":24487,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":134853,"ongoing_decode_tokens":0,"pending_prefill_tokens":72901,"num_requests":2,"active_p_offloads":0,"cached_blocks":1257,"cache_hit":0,"new_prefill":24487,"score_linear":134853.0,"score_lmetric":194776},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":608,"cache_hit":0,"new_prefill":24487,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":16960,"ongoing_decode_tokens":16960,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":801,"cache_hit":0,"new_prefill":24487,"score_linear":16960.0,"score_lmetric":24487},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":172648,"ongoing_decode_tokens":172648,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":795,"cache_hit":0,"new_prefill":24487,"score_linear":172648.0,"score_lmetric":48974}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348584.185388439,"t_first_token_unix":1779779449.1993327,"t_done":348584.479970829,"t_done_unix":1779779449.4939144},{"request_id":"1317160:1:1317160:332","session_id":"1317160","input_length":3441,"t_proxy_recv":348584.612893563,"t_decision_unix":1779779449.6268349,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":3441,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":734,"cache_hit":0,"new_prefill":3441,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":947,"cache_hit":0,"new_prefill":3441,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":898,"cache_hit":0,"new_prefill":3441,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":897,"cache_hit":0,"new_prefill":3441,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":134853,"ongoing_decode_tokens":0,"pending_prefill_tokens":72901,"num_requests":2,"active_p_offloads":0,"cached_blocks":1257,"cache_hit":0,"new_prefill":3441,"score_linear":134853.0,"score_lmetric":152684},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":655,"cache_hit":0,"new_prefill":3441,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":3441,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":176419,"ongoing_decode_tokens":176419,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":810,"cache_hit":0,"new_prefill":3441,"score_linear":176419.0,"score_lmetric":6882}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348584.857590389,"t_first_token_unix":1779779449.8715336,"t_done":348585.014636398,"t_done_unix":1779779450.0285795},{"request_id":"1316848:2:1317438:333","session_id":"1316848","input_length":6040,"t_proxy_recv":348585.693887645,"t_decision_unix":1779779450.7078288,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":5632,"affinity_cache_ratio":0.9324503311258279,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":5632,"estimated_new_tokens":408,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":734,"cache_hit":0,"new_prefill":6040,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":947,"cache_hit":5632,"new_prefill":408,"score_linear":-5632.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":898,"cache_hit":0,"new_prefill":6040,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":903,"cache_hit":0,"new_prefill":6040,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":134853,"ongoing_decode_tokens":0,"pending_prefill_tokens":72901,"num_requests":2,"active_p_offloads":0,"cached_blocks":1257,"cache_hit":1024,"new_prefill":5016,"score_linear":133829.0,"score_lmetric":155834},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":655,"cache_hit":0,"new_prefill":6040,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":6040,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":176419,"ongoing_decode_tokens":176419,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":810,"cache_hit":0,"new_prefill":6040,"score_linear":176419.0,"score_lmetric":12080}],"chosen_score_linear":-5632.0,"chosen_score_lmetric":0,"t_first_token":348585.723553257,"t_first_token_unix":1779779450.7374964,"t_done":348586.134770867,"t_done_unix":1779779451.148714},{"request_id":"1317160:2:1317773:336","session_id":"1317160","input_length":7440,"t_proxy_recv":348587.010996452,"t_decision_unix":1779779452.0249376,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":3072,"affinity_cache_ratio":0.4129032258064516,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":3072,"estimated_new_tokens":4368,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":734,"cache_hit":0,"new_prefill":7440,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":947,"cache_hit":0,"new_prefill":7440,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":898,"cache_hit":0,"new_prefill":7440,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":903,"cache_hit":3072,"new_prefill":4368,"score_linear":-3072.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":134853,"ongoing_decode_tokens":57051,"pending_prefill_tokens":15850,"num_requests":2,"active_p_offloads":0,"cached_blocks":1257,"cache_hit":0,"new_prefill":7440,"score_linear":134853.0,"score_lmetric":46580},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":79663,"ongoing_decode_tokens":0,"pending_prefill_tokens":79663,"num_requests":1,"active_p_offloads":0,"cached_blocks":655,"cache_hit":0,"new_prefill":7440,"score_linear":79663.0,"score_lmetric":87103},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":7440,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":176419,"ongoing_decode_tokens":176419,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":810,"cache_hit":0,"new_prefill":7440,"score_linear":176419.0,"score_lmetric":14880}],"chosen_score_linear":-3072.0,"chosen_score_lmetric":0,"t_first_token":348587.341208035,"t_first_token_unix":1779779452.3551514,"t_done":348587.46884402,"t_done_unix":1779779452.4827874},{"request_id":"1318031:1:1318031:338","session_id":"1318031","input_length":5219,"t_proxy_recv":348587.673613885,"t_decision_unix":1779779452.6875553,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":5219,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":734,"cache_hit":0,"new_prefill":5219,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":947,"cache_hit":0,"new_prefill":5219,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":898,"cache_hit":0,"new_prefill":5219,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":911,"cache_hit":0,"new_prefill":5219,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":134853,"ongoing_decode_tokens":57051,"pending_prefill_tokens":15850,"num_requests":2,"active_p_offloads":0,"cached_blocks":1257,"cache_hit":0,"new_prefill":5219,"score_linear":134853.0,"score_lmetric":42138},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":104431,"ongoing_decode_tokens":0,"pending_prefill_tokens":80367,"num_requests":2,"active_p_offloads":0,"cached_blocks":655,"cache_hit":0,"new_prefill":5219,"score_linear":104431.0,"score_lmetric":171172},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":5219,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":176419,"ongoing_decode_tokens":176419,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":810,"cache_hit":0,"new_prefill":5219,"score_linear":176419.0,"score_lmetric":10438}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348588.010093263,"t_first_token_unix":1779779453.0240371,"t_done":348588.233144593,"t_done_unix":1779779453.2470882},{"request_id":"1318036:1:1318036:339","session_id":"1318036","input_length":2696,"t_proxy_recv":348587.677762818,"t_decision_unix":1779779452.6917045,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":2696,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":734,"cache_hit":0,"new_prefill":2696,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":947,"cache_hit":0,"new_prefill":2696,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":5219,"ongoing_decode_tokens":0,"pending_prefill_tokens":5219,"num_requests":1,"active_p_offloads":0,"cached_blocks":898,"cache_hit":0,"new_prefill":2696,"score_linear":5219.0,"score_lmetric":7915},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":911,"cache_hit":0,"new_prefill":2696,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":134853,"ongoing_decode_tokens":57051,"pending_prefill_tokens":15850,"num_requests":2,"active_p_offloads":0,"cached_blocks":1257,"cache_hit":0,"new_prefill":2696,"score_linear":134853.0,"score_lmetric":37092},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":104431,"ongoing_decode_tokens":0,"pending_prefill_tokens":80367,"num_requests":2,"active_p_offloads":0,"cached_blocks":655,"cache_hit":0,"new_prefill":2696,"score_linear":104431.0,"score_lmetric":166126},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":2696,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":176419,"ongoing_decode_tokens":176419,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":810,"cache_hit":0,"new_prefill":2696,"score_linear":176419.0,"score_lmetric":5392}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348587.845251625,"t_first_token_unix":1779779452.8591955,"t_done":348588.337096171,"t_done_unix":1779779453.3510394},{"request_id":"1310590:2:1315212:324","session_id":"1310590","input_length":32440,"t_proxy_recv":348577.779686431,"t_decision_unix":1779779442.7936277,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":17920,"affinity_cache_ratio":0.5524044389642416,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":17920,"estimated_new_tokens":14520,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":24306,"ongoing_decode_tokens":24306,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":685,"cache_hit":0,"new_prefill":32440,"score_linear":24306.0,"score_lmetric":32440},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":936,"cache_hit":0,"new_prefill":32440,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":898,"cache_hit":0,"new_prefill":32440,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":897,"cache_hit":0,"new_prefill":32440,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":102413,"ongoing_decode_tokens":0,"pending_prefill_tokens":58381,"num_requests":1,"active_p_offloads":0,"cached_blocks":1257,"cache_hit":17920,"new_prefill":14520,"score_linear":84493.0,"score_lmetric":72901},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":608,"cache_hit":0,"new_prefill":32440,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":801,"cache_hit":0,"new_prefill":32440,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":172648,"ongoing_decode_tokens":172648,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":795,"cache_hit":0,"new_prefill":32440,"score_linear":172648.0,"score_lmetric":64880}],"chosen_score_linear":84493.0,"chosen_score_lmetric":72901,"t_first_token":348589.802540998,"t_first_token_unix":1779779454.816485,"t_done":348590.124581396,"t_done_unix":1779779455.1385248},{"request_id":"1269373:9:1300958:261","session_id":"1269373","input_length":54451,"t_proxy_recv":348582.050082907,"t_decision_unix":1779779447.0640247,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":50688,"affinity_cache_ratio":0.9308919946373804,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":50688,"estimated_new_tokens":3763,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":88046,"ongoing_decode_tokens":88046,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":688,"cache_hit":7680,"new_prefill":46771,"score_linear":80366.0,"score_lmetric":93542},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":936,"cache_hit":0,"new_prefill":54451,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":898,"cache_hit":0,"new_prefill":54451,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":897,"cache_hit":0,"new_prefill":54451,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":134853,"ongoing_decode_tokens":0,"pending_prefill_tokens":72901,"num_requests":2,"active_p_offloads":0,"cached_blocks":1257,"cache_hit":0,"new_prefill":54451,"score_linear":134853.0,"score_lmetric":254704},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":24487,"ongoing_decode_tokens":0,"pending_prefill_tokens":24487,"num_requests":1,"active_p_offloads":0,"cached_blocks":608,"cache_hit":0,"new_prefill":54451,"score_linear":24487.0,"score_lmetric":78938},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":16960,"ongoing_decode_tokens":16960,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":801,"cache_hit":7680,"new_prefill":46771,"score_linear":9280.0,"score_lmetric":46771},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":810,"cache_hit":50688,"new_prefill":3763,"score_linear":71280.0,"score_lmetric":3763}],"chosen_score_linear":71280.0,"chosen_score_lmetric":3763,"t_first_token":348583.271547043,"t_first_token_unix":1779779448.2854912,"t_done":348590.435857936,"t_done_unix":1779779455.4498017},{"request_id":"1318604:1:1318604:341","session_id":"1318604","input_length":14412,"t_proxy_recv":348589.722363937,"t_decision_unix":1779779454.7363052,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":14412,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":739,"cache_hit":0,"new_prefill":14412,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":0,"pending_prefill_tokens":3738,"num_requests":1,"active_p_offloads":0,"cached_blocks":947,"cache_hit":0,"new_prefill":14412,"score_linear":62618.0,"score_lmetric":18150},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":908,"cache_hit":0,"new_prefill":14412,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":911,"cache_hit":0,"new_prefill":14412,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":134853,"ongoing_decode_tokens":102413,"pending_prefill_tokens":14520,"num_requests":2,"active_p_offloads":0,"cached_blocks":1257,"cache_hit":0,"new_prefill":14412,"score_linear":134853.0,"score_lmetric":57864},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":104431,"ongoing_decode_tokens":0,"pending_prefill_tokens":80367,"num_requests":2,"active_p_offloads":0,"cached_blocks":655,"cache_hit":0,"new_prefill":14412,"score_linear":104431.0,"score_lmetric":189558},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":14412,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":176419,"ongoing_decode_tokens":176419,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":810,"cache_hit":0,"new_prefill":14412,"score_linear":176419.0,"score_lmetric":28824}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348590.974800881,"t_first_token_unix":1779779455.9887445,"t_done":348591.164722782,"t_done_unix":1779779456.1786659},{"request_id":"1294611:10:1314495:322","session_id":"1294611","input_length":45362,"t_proxy_recv":348575.474511311,"t_decision_unix":1779779440.4884527,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":44032,"affinity_cache_ratio":0.970680305101186,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":44032,"estimated_new_tokens":1330,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":685,"cache_hit":0,"new_prefill":45362,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":936,"cache_hit":0,"new_prefill":45362,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":898,"cache_hit":0,"new_prefill":45362,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":897,"cache_hit":0,"new_prefill":45362,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":57051,"ongoing_decode_tokens":0,"pending_prefill_tokens":57051,"num_requests":0,"active_p_offloads":0,"cached_blocks":1257,"cache_hit":44032,"new_prefill":1330,"score_linear":13019.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":608,"cache_hit":0,"new_prefill":45362,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":801,"cache_hit":0,"new_prefill":45362,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":172648,"ongoing_decode_tokens":50680,"pending_prefill_tokens":24176,"num_requests":2,"active_p_offloads":0,"cached_blocks":795,"cache_hit":12800,"new_prefill":32562,"score_linear":159848.0,"score_lmetric":113476}],"chosen_score_linear":13019.0,"chosen_score_lmetric":0,"t_first_token":348588.400823545,"t_first_token_unix":1779779453.414767,"t_done":348592.111428038,"t_done_unix":1779779457.125372},{"request_id":"1314357:1:1314357:321","session_id":"1314357","input_length":57051,"t_proxy_recv":348575.01534418,"t_decision_unix":1779779440.0292854,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":-57051,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":57051,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":685,"cache_hit":0,"new_prefill":57051,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":936,"cache_hit":0,"new_prefill":57051,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":898,"cache_hit":0,"new_prefill":57051,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":897,"cache_hit":0,"new_prefill":57051,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1257,"cache_hit":0,"new_prefill":57051,"score_linear":0.0,"score_lmetric":-57051},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":608,"cache_hit":3584,"new_prefill":53467,"score_linear":-3584.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":801,"cache_hit":0,"new_prefill":57051,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":172648,"ongoing_decode_tokens":50680,"pending_prefill_tokens":24176,"num_requests":2,"active_p_offloads":0,"cached_blocks":795,"cache_hit":0,"new_prefill":57051,"score_linear":172648.0,"score_lmetric":162454}],"chosen_score_linear":0.0,"chosen_score_lmetric":-57051,"t_first_token":348586.926612458,"t_first_token_unix":1779779451.9405565,"t_done":348592.453916564,"t_done_unix":1779779457.4678605},{"request_id":"1318604:2:1319895:347","session_id":"1318604","input_length":15745,"t_proxy_recv":348594.234998006,"t_decision_unix":1779779459.2489393,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":14336,"affinity_cache_ratio":0.9105112734201334,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":14336,"estimated_new_tokens":1409,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":71854,"ongoing_decode_tokens":0,"pending_prefill_tokens":6830,"num_requests":1,"active_p_offloads":0,"cached_blocks":739,"cache_hit":0,"new_prefill":15745,"score_linear":71854.0,"score_lmetric":22575},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":68916,"ongoing_decode_tokens":0,"pending_prefill_tokens":4404,"num_requests":2,"active_p_offloads":0,"cached_blocks":947,"cache_hit":0,"new_prefill":15745,"score_linear":68916.0,"score_lmetric":40298},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":936,"cache_hit":14336,"new_prefill":1409,"score_linear":-14336.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":911,"cache_hit":0,"new_prefill":15745,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1398,"cache_hit":0,"new_prefill":15745,"score_linear":0.0,"score_lmetric":-15745},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":104431,"ongoing_decode_tokens":0,"pending_prefill_tokens":80367,"num_requests":2,"active_p_offloads":0,"cached_blocks":655,"cache_hit":0,"new_prefill":15745,"score_linear":104431.0,"score_lmetric":192224},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":15745,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":181114,"ongoing_decode_tokens":181114,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":818,"cache_hit":0,"new_prefill":15745,"score_linear":181114.0,"score_lmetric":31490}],"chosen_score_linear":-14336.0,"chosen_score_lmetric":0,"t_first_token":348594.521772071,"t_first_token_unix":1779779459.5357153,"t_done":348594.708882746,"t_done_unix":1779779459.7228265},{"request_id":"1269373:10:1304899:282","session_id":"1269373","input_length":59146,"t_proxy_recv":348590.449888672,"t_decision_unix":1779779455.4638302,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":54784,"affinity_cache_ratio":0.9262502958779968,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":54784,"estimated_new_tokens":4362,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":739,"cache_hit":7680,"new_prefill":51466,"score_linear":-7680.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":0,"pending_prefill_tokens":3738,"num_requests":1,"active_p_offloads":0,"cached_blocks":947,"cache_hit":0,"new_prefill":59146,"score_linear":62618.0,"score_lmetric":62884},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":14412,"ongoing_decode_tokens":0,"pending_prefill_tokens":14412,"num_requests":1,"active_p_offloads":0,"cached_blocks":908,"cache_hit":0,"new_prefill":59146,"score_linear":14412.0,"score_lmetric":73558},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":911,"cache_hit":0,"new_prefill":59146,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":102413,"ongoing_decode_tokens":102413,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1285,"cache_hit":0,"new_prefill":59146,"score_linear":102413.0,"score_lmetric":59146},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":104431,"ongoing_decode_tokens":0,"pending_prefill_tokens":80367,"num_requests":2,"active_p_offloads":0,"cached_blocks":655,"cache_hit":0,"new_prefill":59146,"score_linear":104431.0,"score_lmetric":279026},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":820,"cache_hit":7680,"new_prefill":51466,"score_linear":-7680.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":818,"cache_hit":54784,"new_prefill":4362,"score_linear":67184.0,"score_lmetric":4362}],"chosen_score_linear":67184.0,"chosen_score_lmetric":4362,"t_first_token":348592.088304246,"t_first_token_unix":1779779457.1022482,"t_done":348595.844856397,"t_done_unix":1779779460.8588004},{"request_id":"1279412:14:1319059:344","session_id":"1279412","input_length":71854,"t_proxy_recv":348591.455582197,"t_decision_unix":1779779456.4695237,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":65024,"affinity_cache_ratio":0.9049461407854816,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":65024,"estimated_new_tokens":6830,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":739,"cache_hit":65024,"new_prefill":6830,"score_linear":-65024.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":68916,"ongoing_decode_tokens":0,"pending_prefill_tokens":4404,"num_requests":2,"active_p_offloads":0,"cached_blocks":947,"cache_hit":0,"new_prefill":71854,"score_linear":68916.0,"score_lmetric":152516},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":936,"cache_hit":0,"new_prefill":71854,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":911,"cache_hit":0,"new_prefill":71854,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":102413,"ongoing_decode_tokens":102413,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1285,"cache_hit":0,"new_prefill":71854,"score_linear":102413.0,"score_lmetric":71854},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":104431,"ongoing_decode_tokens":0,"pending_prefill_tokens":80367,"num_requests":2,"active_p_offloads":0,"cached_blocks":655,"cache_hit":0,"new_prefill":71854,"score_linear":104431.0,"score_lmetric":304442},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":820,"cache_hit":12800,"new_prefill":59054,"score_linear":-12800.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":181114,"ongoing_decode_tokens":121968,"pending_prefill_tokens":4362,"num_requests":2,"active_p_offloads":0,"cached_blocks":818,"cache_hit":12800,"new_prefill":59054,"score_linear":168314.0,"score_lmetric":126832}],"chosen_score_linear":-65024.0,"chosen_score_lmetric":0,"t_first_token":348594.434555276,"t_first_token_unix":1779779459.4484992,"t_done":348597.315865643,"t_done_unix":1779779462.3298092},{"request_id":"1318604:3:1320865:350","session_id":"1318604","input_length":17883,"t_proxy_recv":348597.855965,"t_decision_unix":1779779462.8699064,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":15360,"affinity_cache_ratio":0.8589162892132193,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":15360,"estimated_new_tokens":2523,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":752,"cache_hit":0,"new_prefill":17883,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":68916,"ongoing_decode_tokens":0,"pending_prefill_tokens":4404,"num_requests":2,"active_p_offloads":0,"cached_blocks":947,"cache_hit":0,"new_prefill":17883,"score_linear":68916.0,"score_lmetric":44574},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":938,"cache_hit":15360,"new_prefill":2523,"score_linear":-15360.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":911,"cache_hit":0,"new_prefill":17883,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":57645,"ongoing_decode_tokens":0,"pending_prefill_tokens":57645,"num_requests":1,"active_p_offloads":0,"cached_blocks":1398,"cache_hit":0,"new_prefill":17883,"score_linear":57645.0,"score_lmetric":75528},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":104431,"ongoing_decode_tokens":0,"pending_prefill_tokens":80367,"num_requests":2,"active_p_offloads":0,"cached_blocks":655,"cache_hit":0,"new_prefill":17883,"score_linear":104431.0,"score_lmetric":196500},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":17883,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":181414,"ongoing_decode_tokens":181414,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":826,"cache_hit":0,"new_prefill":17883,"score_linear":181414.0,"score_lmetric":35766}],"chosen_score_linear":-15360.0,"chosen_score_lmetric":0,"t_first_token":348598.20312276,"t_first_token_unix":1779779463.2170665,"t_done":348598.429936767,"t_done_unix":1779779463.4438806},{"request_id":"1318036:2:1321282:353","session_id":"1318036","input_length":2677,"t_proxy_recv":348599.317641825,"t_decision_unix":1779779464.3315833,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":2560,"affinity_cache_ratio":0.9562943593574897,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":2560,"estimated_new_tokens":117,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":752,"cache_hit":2560,"new_prefill":117,"score_linear":-2560.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":68916,"ongoing_decode_tokens":0,"pending_prefill_tokens":4404,"num_requests":2,"active_p_offloads":0,"cached_blocks":947,"cache_hit":0,"new_prefill":2677,"score_linear":68916.0,"score_lmetric":14162},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":2677,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":911,"cache_hit":0,"new_prefill":2677,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":57645,"ongoing_decode_tokens":0,"pending_prefill_tokens":57645,"num_requests":1,"active_p_offloads":0,"cached_blocks":1398,"cache_hit":0,"new_prefill":2677,"score_linear":57645.0,"score_lmetric":60322},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":138337,"ongoing_decode_tokens":0,"pending_prefill_tokens":83041,"num_requests":3,"active_p_offloads":0,"cached_blocks":655,"cache_hit":0,"new_prefill":2677,"score_linear":138337.0,"score_lmetric":257154},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":8095,"ongoing_decode_tokens":8095,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":2677,"score_linear":8095.0,"score_lmetric":2677},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":181414,"ongoing_decode_tokens":181414,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":826,"cache_hit":0,"new_prefill":2677,"score_linear":181414.0,"score_lmetric":5354}],"chosen_score_linear":-2560.0,"chosen_score_lmetric":0,"t_first_token":348599.337866561,"t_first_token_unix":1779779464.35181,"t_done":348599.345824357,"t_done_unix":1779779464.3597672},{"request_id":"1321042:1:1321042:351","session_id":"1321042","input_length":8095,"t_proxy_recv":348598.512641326,"t_decision_unix":1779779463.5265822,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":8095,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":752,"cache_hit":0,"new_prefill":8095,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":68916,"ongoing_decode_tokens":0,"pending_prefill_tokens":4404,"num_requests":2,"active_p_offloads":0,"cached_blocks":947,"cache_hit":0,"new_prefill":8095,"score_linear":68916.0,"score_lmetric":24998},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":8095,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":911,"cache_hit":0,"new_prefill":8095,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":57645,"ongoing_decode_tokens":0,"pending_prefill_tokens":57645,"num_requests":1,"active_p_offloads":0,"cached_blocks":1398,"cache_hit":0,"new_prefill":8095,"score_linear":57645.0,"score_lmetric":65740},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":104431,"ongoing_decode_tokens":0,"pending_prefill_tokens":80367,"num_requests":2,"active_p_offloads":0,"cached_blocks":655,"cache_hit":0,"new_prefill":8095,"score_linear":104431.0,"score_lmetric":176924},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":8095,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":181414,"ongoing_decode_tokens":181414,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":826,"cache_hit":0,"new_prefill":8095,"score_linear":181414.0,"score_lmetric":16190}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348599.091284436,"t_first_token_unix":1779779464.1052282,"t_done":348599.40193715,"t_done_unix":1779779464.415881},{"request_id":"1316848:3:1318975:343","session_id":"1316848","input_length":6298,"t_proxy_recv":348591.094070327,"t_decision_unix":1779779456.1080115,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":5632,"affinity_cache_ratio":0.8942521435376309,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":5632,"estimated_new_tokens":666,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":739,"cache_hit":0,"new_prefill":6298,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":0,"pending_prefill_tokens":3738,"num_requests":1,"active_p_offloads":0,"cached_blocks":947,"cache_hit":5632,"new_prefill":666,"score_linear":56986.0,"score_lmetric":4404},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":14412,"ongoing_decode_tokens":14412,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":908,"cache_hit":0,"new_prefill":6298,"score_linear":14412.0,"score_lmetric":6298},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":911,"cache_hit":0,"new_prefill":6298,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":102413,"ongoing_decode_tokens":102413,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1285,"cache_hit":1024,"new_prefill":5274,"score_linear":101389.0,"score_lmetric":5274},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":104431,"ongoing_decode_tokens":0,"pending_prefill_tokens":80367,"num_requests":2,"active_p_offloads":0,"cached_blocks":655,"cache_hit":0,"new_prefill":6298,"score_linear":104431.0,"score_lmetric":173330},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":6298,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":181114,"ongoing_decode_tokens":121968,"pending_prefill_tokens":4362,"num_requests":2,"active_p_offloads":0,"cached_blocks":818,"cache_hit":0,"new_prefill":6298,"score_linear":181114.0,"score_lmetric":21320}],"chosen_score_linear":56986.0,"chosen_score_lmetric":4404,"t_first_token":348599.661411345,"t_first_token_unix":1779779464.6753547,"t_done":348600.12931976,"t_done_unix":1779779465.1432633},{"request_id":"1318604:4:1321889:356","session_id":"1318604","input_length":21033,"t_proxy_recv":348601.447963717,"t_decision_unix":1779779466.461905,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":17920,"affinity_cache_ratio":0.8519944848571293,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":17920,"estimated_new_tokens":3113,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":93542,"ongoing_decode_tokens":0,"pending_prefill_tokens":21862,"num_requests":2,"active_p_offloads":0,"cached_blocks":752,"cache_hit":0,"new_prefill":21033,"score_linear":93542.0,"score_lmetric":85790},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":948,"cache_hit":0,"new_prefill":21033,"score_linear":62618.0,"score_lmetric":21033},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":943,"cache_hit":17920,"new_prefill":3113,"score_linear":-17920.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":911,"cache_hit":0,"new_prefill":21033,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":57645,"ongoing_decode_tokens":0,"pending_prefill_tokens":57645,"num_requests":1,"active_p_offloads":0,"cached_blocks":1398,"cache_hit":0,"new_prefill":21033,"score_linear":57645.0,"score_lmetric":78678},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":138337,"ongoing_decode_tokens":0,"pending_prefill_tokens":83041,"num_requests":3,"active_p_offloads":0,"cached_blocks":655,"cache_hit":0,"new_prefill":21033,"score_linear":138337.0,"score_lmetric":312222},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":835,"cache_hit":0,"new_prefill":21033,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":181414,"ongoing_decode_tokens":181414,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":826,"cache_hit":0,"new_prefill":21033,"score_linear":181414.0,"score_lmetric":42066}],"chosen_score_linear":-17920.0,"chosen_score_lmetric":0,"t_first_token":348601.971243218,"t_first_token_unix":1779779466.9851868,"t_done":348602.212128169,"t_done_unix":1779779467.2260716},{"request_id":"1269373:11:1307381:296","session_id":"1269373","input_length":59446,"t_proxy_recv":348595.861112781,"t_decision_unix":1779779460.8750544,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":58880,"affinity_cache_ratio":0.9904787538270027,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":58880,"estimated_new_tokens":566,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":71854,"ongoing_decode_tokens":71854,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":739,"cache_hit":7680,"new_prefill":51766,"score_linear":64174.0,"score_lmetric":51766},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":68916,"ongoing_decode_tokens":0,"pending_prefill_tokens":4404,"num_requests":2,"active_p_offloads":0,"cached_blocks":947,"cache_hit":0,"new_prefill":59446,"score_linear":68916.0,"score_lmetric":127700},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":938,"cache_hit":0,"new_prefill":59446,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":911,"cache_hit":0,"new_prefill":59446,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":54546,"ongoing_decode_tokens":0,"pending_prefill_tokens":54546,"num_requests":0,"active_p_offloads":0,"cached_blocks":1398,"cache_hit":0,"new_prefill":59446,"score_linear":54546.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":104431,"ongoing_decode_tokens":0,"pending_prefill_tokens":80367,"num_requests":2,"active_p_offloads":0,"cached_blocks":655,"cache_hit":0,"new_prefill":59446,"score_linear":104431.0,"score_lmetric":279626},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":820,"cache_hit":7680,"new_prefill":51766,"score_linear":-7680.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":826,"cache_hit":58880,"new_prefill":566,"score_linear":63088.0,"score_lmetric":566}],"chosen_score_linear":63088.0,"chosen_score_lmetric":566,"t_first_token":348596.018228434,"t_first_token_unix":1779779461.0321722,"t_done":348604.009666377,"t_done_unix":1779779469.0236096},{"request_id":"1322540:1:1322540:359","session_id":"1322540","input_length":8939,"t_proxy_recv":348603.908783249,"t_decision_unix":1779779468.9227242,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":8939,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":93542,"ongoing_decode_tokens":93542,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":752,"cache_hit":0,"new_prefill":8939,"score_linear":93542.0,"score_lmetric":17878},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":948,"cache_hit":0,"new_prefill":8939,"score_linear":62618.0,"score_lmetric":8939},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":949,"cache_hit":0,"new_prefill":8939,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":911,"cache_hit":0,"new_prefill":8939,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":57645,"ongoing_decode_tokens":0,"pending_prefill_tokens":57645,"num_requests":1,"active_p_offloads":0,"cached_blocks":1398,"cache_hit":0,"new_prefill":8939,"score_linear":57645.0,"score_lmetric":66584},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":138337,"ongoing_decode_tokens":0,"pending_prefill_tokens":83041,"num_requests":3,"active_p_offloads":0,"cached_blocks":655,"cache_hit":0,"new_prefill":8939,"score_linear":138337.0,"score_lmetric":275940},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":33182,"ongoing_decode_tokens":0,"pending_prefill_tokens":33182,"num_requests":1,"active_p_offloads":0,"cached_blocks":835,"cache_hit":0,"new_prefill":8939,"score_linear":33182.0,"score_lmetric":42121},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":181414,"ongoing_decode_tokens":181414,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":826,"cache_hit":0,"new_prefill":8939,"score_linear":181414.0,"score_lmetric":17878}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348604.603227155,"t_first_token_unix":1779779469.6171713,"t_done":348604.866696122,"t_done_unix":1779779469.8806396},{"request_id":"1318031:2:1322552:360","session_id":"1318031","input_length":11145,"t_proxy_recv":348603.961989127,"t_decision_unix":1779779468.9759305,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":2,"chosen_idx":3,"affinity_cache_hit":5120,"affinity_cache_ratio":0.45939883355764916,"affinity_num_requests":1,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":11145,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":93542,"ongoing_decode_tokens":93542,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":752,"cache_hit":0,"new_prefill":11145,"score_linear":93542.0,"score_lmetric":22290},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":948,"cache_hit":0,"new_prefill":11145,"score_linear":62618.0,"score_lmetric":11145},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":8939,"ongoing_decode_tokens":0,"pending_prefill_tokens":8939,"num_requests":1,"active_p_offloads":0,"cached_blocks":949,"cache_hit":5120,"new_prefill":6025,"score_linear":3819.0,"score_lmetric":14964},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":911,"cache_hit":0,"new_prefill":11145,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":57645,"ongoing_decode_tokens":0,"pending_prefill_tokens":57645,"num_requests":1,"active_p_offloads":0,"cached_blocks":1398,"cache_hit":0,"new_prefill":11145,"score_linear":57645.0,"score_lmetric":68790},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":138337,"ongoing_decode_tokens":0,"pending_prefill_tokens":83041,"num_requests":3,"active_p_offloads":0,"cached_blocks":655,"cache_hit":0,"new_prefill":11145,"score_linear":138337.0,"score_lmetric":282558},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":33182,"ongoing_decode_tokens":0,"pending_prefill_tokens":33182,"num_requests":1,"active_p_offloads":0,"cached_blocks":835,"cache_hit":0,"new_prefill":11145,"score_linear":33182.0,"score_lmetric":44327},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":181414,"ongoing_decode_tokens":181414,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":826,"cache_hit":0,"new_prefill":11145,"score_linear":181414.0,"score_lmetric":22290}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348604.850020452,"t_first_token_unix":1779779469.8639638,"t_done":348604.899120249,"t_done_unix":1779779469.9130635},{"request_id":"1279412:15:1321613:355","session_id":"1279412","input_length":72402,"t_proxy_recv":348600.43727336,"t_decision_unix":1779779465.451214,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":71680,"affinity_cache_ratio":0.990027899781774,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":71680,"estimated_new_tokens":722,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":21140,"ongoing_decode_tokens":0,"pending_prefill_tokens":21140,"num_requests":1,"active_p_offloads":0,"cached_blocks":752,"cache_hit":71680,"new_prefill":722,"score_linear":-50540.0,"score_lmetric":21862},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":948,"cache_hit":0,"new_prefill":72402,"score_linear":62618.0,"score_lmetric":72402},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":72402,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":911,"cache_hit":0,"new_prefill":72402,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":57645,"ongoing_decode_tokens":0,"pending_prefill_tokens":57645,"num_requests":1,"active_p_offloads":0,"cached_blocks":1398,"cache_hit":0,"new_prefill":72402,"score_linear":57645.0,"score_lmetric":130047},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":138337,"ongoing_decode_tokens":0,"pending_prefill_tokens":83041,"num_requests":3,"active_p_offloads":0,"cached_blocks":655,"cache_hit":0,"new_prefill":72402,"score_linear":138337.0,"score_lmetric":466329},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":835,"cache_hit":12800,"new_prefill":59602,"score_linear":-12800.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":181414,"ongoing_decode_tokens":181414,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":826,"cache_hit":12800,"new_prefill":59602,"score_linear":168614.0,"score_lmetric":119204}],"chosen_score_linear":-50540.0,"chosen_score_lmetric":21862,"t_first_token":348602.556094878,"t_first_token_unix":1779779467.5700383,"t_done":348605.311433295,"t_done_unix":1779779470.3253767},{"request_id":"1316848:4:1319820:345","session_id":"1316848","input_length":21140,"t_proxy_recv":348600.140511899,"t_decision_unix":1779779465.1544535,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":1,"chosen_idx":0,"affinity_cache_hit":6144,"affinity_cache_ratio":0.29063386944181646,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":21140,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":752,"cache_hit":0,"new_prefill":21140,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":948,"cache_hit":6144,"new_prefill":14996,"score_linear":56474.0,"score_lmetric":14996},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":21140,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":911,"cache_hit":0,"new_prefill":21140,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":57645,"ongoing_decode_tokens":0,"pending_prefill_tokens":57645,"num_requests":1,"active_p_offloads":0,"cached_blocks":1398,"cache_hit":1024,"new_prefill":20116,"score_linear":56621.0,"score_lmetric":77761},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":138337,"ongoing_decode_tokens":0,"pending_prefill_tokens":83041,"num_requests":3,"active_p_offloads":0,"cached_blocks":655,"cache_hit":0,"new_prefill":21140,"score_linear":138337.0,"score_lmetric":312543},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":835,"cache_hit":0,"new_prefill":21140,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":181414,"ongoing_decode_tokens":181414,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":826,"cache_hit":0,"new_prefill":21140,"score_linear":181414.0,"score_lmetric":42280}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348602.552201083,"t_first_token_unix":1779779467.5661464,"t_done":348605.600616026,"t_done_unix":1779779470.61456},{"request_id":"1320630:1:1320630:348","session_id":"1320630","input_length":3099,"t_proxy_recv":348597.012618732,"t_decision_unix":1779779462.02656,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":3099,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":71854,"ongoing_decode_tokens":71854,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":739,"cache_hit":0,"new_prefill":3099,"score_linear":71854.0,"score_lmetric":3099},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":68916,"ongoing_decode_tokens":0,"pending_prefill_tokens":4404,"num_requests":2,"active_p_offloads":0,"cached_blocks":947,"cache_hit":0,"new_prefill":3099,"score_linear":68916.0,"score_lmetric":15006},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":938,"cache_hit":0,"new_prefill":3099,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":911,"cache_hit":0,"new_prefill":3099,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":54546,"ongoing_decode_tokens":0,"pending_prefill_tokens":54546,"num_requests":0,"active_p_offloads":0,"cached_blocks":1398,"cache_hit":0,"new_prefill":3099,"score_linear":54546.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":104431,"ongoing_decode_tokens":0,"pending_prefill_tokens":80367,"num_requests":2,"active_p_offloads":0,"cached_blocks":655,"cache_hit":0,"new_prefill":3099,"score_linear":104431.0,"score_lmetric":166932},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":3099,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":181414,"ongoing_decode_tokens":181414,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":826,"cache_hit":0,"new_prefill":3099,"score_linear":181414.0,"score_lmetric":6198}],"chosen_score_linear":54546.0,"chosen_score_lmetric":0,"t_first_token":348605.341963566,"t_first_token_unix":1779779470.3559105,"t_done":348605.681983163,"t_done_unix":1779779470.6959267},{"request_id":"1316848:5:1322192:357","session_id":"1316848","input_length":21471,"t_proxy_recv":348605.606337825,"t_decision_unix":1779779470.6202796,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":20992,"affinity_cache_ratio":0.9776908388058311,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":20992,"estimated_new_tokens":479,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":794,"cache_hit":20992,"new_prefill":479,"score_linear":-20992.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":948,"cache_hit":6144,"new_prefill":15327,"score_linear":56474.0,"score_lmetric":15327},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":966,"cache_hit":0,"new_prefill":21471,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":932,"cache_hit":0,"new_prefill":21471,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":57645,"ongoing_decode_tokens":57645,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1398,"cache_hit":1024,"new_prefill":20447,"score_linear":56621.0,"score_lmetric":20447},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":138337,"ongoing_decode_tokens":0,"pending_prefill_tokens":83041,"num_requests":3,"active_p_offloads":0,"cached_blocks":655,"cache_hit":0,"new_prefill":21471,"score_linear":138337.0,"score_lmetric":313536},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":33182,"ongoing_decode_tokens":0,"pending_prefill_tokens":33182,"num_requests":1,"active_p_offloads":0,"cached_blocks":835,"cache_hit":0,"new_prefill":21471,"score_linear":33182.0,"score_lmetric":54653},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":184808,"ongoing_decode_tokens":184808,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":828,"cache_hit":0,"new_prefill":21471,"score_linear":184808.0,"score_lmetric":42942}],"chosen_score_linear":-20992.0,"chosen_score_lmetric":0,"t_first_token":348605.653538915,"t_first_token_unix":1779779470.6674826,"t_done":348606.192442997,"t_done_unix":1779779471.206387},{"request_id":"1322540:2:1323519:362","session_id":"1322540","input_length":9016,"t_proxy_recv":348607.204682511,"t_decision_unix":1779779472.2186236,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":8704,"affinity_cache_ratio":0.9653948535936113,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":8704,"estimated_new_tokens":312,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":795,"cache_hit":0,"new_prefill":9016,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":948,"cache_hit":0,"new_prefill":9016,"score_linear":62618.0,"score_lmetric":9016},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":966,"cache_hit":8704,"new_prefill":312,"score_linear":-8704.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":932,"cache_hit":0,"new_prefill":9016,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":54546,"ongoing_decode_tokens":54546,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1404,"cache_hit":0,"new_prefill":9016,"score_linear":54546.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":138337,"ongoing_decode_tokens":0,"pending_prefill_tokens":83041,"num_requests":3,"active_p_offloads":0,"cached_blocks":655,"cache_hit":0,"new_prefill":9016,"score_linear":138337.0,"score_lmetric":276171},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":46190,"ongoing_decode_tokens":0,"pending_prefill_tokens":38510,"num_requests":2,"active_p_offloads":0,"cached_blocks":835,"cache_hit":0,"new_prefill":9016,"score_linear":46190.0,"score_lmetric":95052},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":184808,"ongoing_decode_tokens":184808,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":828,"cache_hit":0,"new_prefill":9016,"score_linear":184808.0,"score_lmetric":18032}],"chosen_score_linear":-8704.0,"chosen_score_lmetric":0,"t_first_token":348607.23979113,"t_first_token_unix":1779779472.2537344,"t_done":348607.366431898,"t_done_unix":1779779472.3803754},{"request_id":"1321042:2:1323492:361","session_id":"1321042","input_length":13008,"t_proxy_recv":348607.135934374,"t_decision_unix":1779779472.1498754,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":7680,"affinity_cache_ratio":0.5904059040590406,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":7680,"estimated_new_tokens":5328,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":795,"cache_hit":0,"new_prefill":13008,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":948,"cache_hit":0,"new_prefill":13008,"score_linear":62618.0,"score_lmetric":13008},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":966,"cache_hit":0,"new_prefill":13008,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":932,"cache_hit":0,"new_prefill":13008,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":54546,"ongoing_decode_tokens":54546,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1404,"cache_hit":0,"new_prefill":13008,"score_linear":54546.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":138337,"ongoing_decode_tokens":0,"pending_prefill_tokens":83041,"num_requests":3,"active_p_offloads":0,"cached_blocks":655,"cache_hit":0,"new_prefill":13008,"score_linear":138337.0,"score_lmetric":288147},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":33182,"ongoing_decode_tokens":0,"pending_prefill_tokens":33182,"num_requests":1,"active_p_offloads":0,"cached_blocks":835,"cache_hit":7680,"new_prefill":5328,"score_linear":25502.0,"score_lmetric":38510},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":184808,"ongoing_decode_tokens":184808,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":828,"cache_hit":0,"new_prefill":13008,"score_linear":184808.0,"score_lmetric":26016}],"chosen_score_linear":25502.0,"chosen_score_lmetric":38510,"t_first_token":348607.956923124,"t_first_token_unix":1779779472.970867,"t_done":348608.520947121,"t_done_unix":1779779473.5348923},{"request_id":"1315567:2:1319894:346","session_id":"1315567","input_length":54546,"t_proxy_recv":348594.240597958,"t_decision_unix":1779779459.2545393,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":6,"chosen_idx":4,"affinity_cache_hit":17408,"affinity_cache_ratio":0.31914347523191433,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":-54546,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":54546,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":71854,"ongoing_decode_tokens":0,"pending_prefill_tokens":6830,"num_requests":1,"active_p_offloads":0,"cached_blocks":739,"cache_hit":7680,"new_prefill":46866,"score_linear":64174.0,"score_lmetric":53696},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":68916,"ongoing_decode_tokens":0,"pending_prefill_tokens":4404,"num_requests":2,"active_p_offloads":0,"cached_blocks":947,"cache_hit":0,"new_prefill":54546,"score_linear":68916.0,"score_lmetric":117900},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":15745,"ongoing_decode_tokens":0,"pending_prefill_tokens":1409,"num_requests":1,"active_p_offloads":0,"cached_blocks":936,"cache_hit":0,"new_prefill":54546,"score_linear":15745.0,"score_lmetric":55955},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":911,"cache_hit":0,"new_prefill":54546,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1398,"cache_hit":0,"new_prefill":54546,"score_linear":0.0,"score_lmetric":-54546},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":104431,"ongoing_decode_tokens":0,"pending_prefill_tokens":80367,"num_requests":2,"active_p_offloads":0,"cached_blocks":655,"cache_hit":0,"new_prefill":54546,"score_linear":104431.0,"score_lmetric":269826},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":820,"cache_hit":17408,"new_prefill":37138,"score_linear":-17408.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":181114,"ongoing_decode_tokens":181114,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":818,"cache_hit":7680,"new_prefill":46866,"score_linear":173434.0,"score_lmetric":93732}],"chosen_score_linear":0.0,"chosen_score_lmetric":-54546,"t_first_token":348605.326236926,"t_first_token_unix":1779779470.3401802,"t_done":348608.944196765,"t_done_unix":1779779473.95814},{"request_id":"1320630:2:1323748:364","session_id":"1320630","input_length":11298,"t_proxy_recv":348607.976594406,"t_decision_unix":1779779472.990536,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":3072,"affinity_cache_ratio":0.27190653212958044,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":false,"cache_hit":3072,"estimated_new_tokens":8226,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":24395,"ongoing_decode_tokens":24395,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":795,"cache_hit":0,"new_prefill":11298,"score_linear":24395.0,"score_lmetric":11298},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":948,"cache_hit":0,"new_prefill":11298,"score_linear":62618.0,"score_lmetric":11298},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":966,"cache_hit":0,"new_prefill":11298,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":932,"cache_hit":0,"new_prefill":11298,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":54546,"ongoing_decode_tokens":54546,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1404,"cache_hit":3072,"new_prefill":8226,"score_linear":51474.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":138337,"ongoing_decode_tokens":0,"pending_prefill_tokens":83041,"num_requests":3,"active_p_offloads":0,"cached_blocks":655,"cache_hit":0,"new_prefill":11298,"score_linear":138337.0,"score_lmetric":283017},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":46190,"ongoing_decode_tokens":46190,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":835,"cache_hit":0,"new_prefill":11298,"score_linear":46190.0,"score_lmetric":22596},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":184808,"ongoing_decode_tokens":184808,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":828,"cache_hit":0,"new_prefill":11298,"score_linear":184808.0,"score_lmetric":22596}],"chosen_score_linear":51474.0,"chosen_score_lmetric":0,"t_first_token":348608.702405497,"t_first_token_unix":1779779473.7163486,"t_done":348609.082717781,"t_done_unix":1779779474.0966616},{"request_id":"1316201:2:1317776:337","session_id":"1316201","input_length":24768,"t_proxy_recv":348587.034880046,"t_decision_unix":1779779452.0488212,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":24064,"affinity_cache_ratio":0.9715762273901809,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":24064,"estimated_new_tokens":704,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":734,"cache_hit":0,"new_prefill":24768,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":947,"cache_hit":0,"new_prefill":24768,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":898,"cache_hit":0,"new_prefill":24768,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":7440,"ongoing_decode_tokens":0,"pending_prefill_tokens":4368,"num_requests":1,"active_p_offloads":0,"cached_blocks":903,"cache_hit":0,"new_prefill":24768,"score_linear":7440.0,"score_lmetric":29136},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":134853,"ongoing_decode_tokens":57051,"pending_prefill_tokens":15850,"num_requests":2,"active_p_offloads":0,"cached_blocks":1257,"cache_hit":0,"new_prefill":24768,"score_linear":134853.0,"score_lmetric":81236},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":79663,"ongoing_decode_tokens":0,"pending_prefill_tokens":79663,"num_requests":1,"active_p_offloads":0,"cached_blocks":655,"cache_hit":24064,"new_prefill":704,"score_linear":55599.0,"score_lmetric":80367},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":24768,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":176419,"ongoing_decode_tokens":176419,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":810,"cache_hit":0,"new_prefill":24768,"score_linear":176419.0,"score_lmetric":49536}],"chosen_score_linear":55599.0,"chosen_score_lmetric":80367,"t_first_token":348608.202964302,"t_first_token_unix":1779779473.2169077,"t_done":348609.943987416,"t_done_unix":1779779474.9579308},{"request_id":"1277428:4:1321259:352","session_id":"1277428","input_length":33906,"t_proxy_recv":348599.264927224,"t_decision_unix":1779779464.2788687,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":31232,"affinity_cache_ratio":0.9211349023771603,"affinity_num_requests":2,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":31232,"estimated_new_tokens":2674,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":752,"cache_hit":0,"new_prefill":33906,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":68916,"ongoing_decode_tokens":0,"pending_prefill_tokens":4404,"num_requests":2,"active_p_offloads":0,"cached_blocks":947,"cache_hit":4608,"new_prefill":29298,"score_linear":64308.0,"score_lmetric":67404},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":943,"cache_hit":0,"new_prefill":33906,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":911,"cache_hit":17408,"new_prefill":16498,"score_linear":-17408.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":57645,"ongoing_decode_tokens":0,"pending_prefill_tokens":57645,"num_requests":1,"active_p_offloads":0,"cached_blocks":1398,"cache_hit":0,"new_prefill":33906,"score_linear":57645.0,"score_lmetric":91551},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":104431,"ongoing_decode_tokens":0,"pending_prefill_tokens":80367,"num_requests":2,"active_p_offloads":0,"cached_blocks":655,"cache_hit":31232,"new_prefill":2674,"score_linear":73199.0,"score_lmetric":166082},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":8095,"ongoing_decode_tokens":8095,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":33906,"score_linear":8095.0,"score_lmetric":33906},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":181414,"ongoing_decode_tokens":181414,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":826,"cache_hit":0,"new_prefill":33906,"score_linear":181414.0,"score_lmetric":67812}],"chosen_score_linear":73199.0,"chosen_score_lmetric":166082,"t_first_token":348608.265023261,"t_first_token_unix":1779779473.2789667,"t_done":348610.039436513,"t_done_unix":1779779475.05338},{"request_id":"1316848:6:1323535:363","session_id":"1316848","input_length":24395,"t_proxy_recv":348607.254599986,"t_decision_unix":1779779472.2685413,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":21504,"affinity_cache_ratio":0.8814921090387374,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":21504,"estimated_new_tokens":2891,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":795,"cache_hit":21504,"new_prefill":2891,"score_linear":-21504.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":948,"cache_hit":6144,"new_prefill":18251,"score_linear":56474.0,"score_lmetric":18251},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":9016,"ongoing_decode_tokens":9016,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":966,"cache_hit":0,"new_prefill":24395,"score_linear":9016.0,"score_lmetric":24395},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":932,"cache_hit":0,"new_prefill":24395,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":54546,"ongoing_decode_tokens":54546,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1404,"cache_hit":1024,"new_prefill":23371,"score_linear":53522.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":138337,"ongoing_decode_tokens":0,"pending_prefill_tokens":83041,"num_requests":3,"active_p_offloads":0,"cached_blocks":655,"cache_hit":0,"new_prefill":24395,"score_linear":138337.0,"score_lmetric":322308},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":46190,"ongoing_decode_tokens":0,"pending_prefill_tokens":38510,"num_requests":2,"active_p_offloads":0,"cached_blocks":835,"cache_hit":0,"new_prefill":24395,"score_linear":46190.0,"score_lmetric":125810},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":184808,"ongoing_decode_tokens":184808,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":828,"cache_hit":0,"new_prefill":24395,"score_linear":184808.0,"score_lmetric":48790}],"chosen_score_linear":-21504.0,"chosen_score_lmetric":0,"t_first_token":348607.822948058,"t_first_token_unix":1779779472.8368921,"t_done":348610.068599998,"t_done_unix":1779779475.082543},{"request_id":"1269373:12:1312836:316","session_id":"1269373","input_length":62840,"t_proxy_recv":348604.01990565,"t_decision_unix":1779779469.033847,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":59904,"affinity_cache_ratio":0.9532781667727562,"affinity_num_requests":1,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":59904,"estimated_new_tokens":2936,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":93542,"ongoing_decode_tokens":93542,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":752,"cache_hit":7680,"new_prefill":55160,"score_linear":85862.0,"score_lmetric":110320},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":948,"cache_hit":0,"new_prefill":62840,"score_linear":62618.0,"score_lmetric":62840},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":8939,"ongoing_decode_tokens":0,"pending_prefill_tokens":8939,"num_requests":1,"active_p_offloads":0,"cached_blocks":949,"cache_hit":0,"new_prefill":62840,"score_linear":8939.0,"score_lmetric":71779},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":11145,"ongoing_decode_tokens":0,"pending_prefill_tokens":11145,"num_requests":1,"active_p_offloads":0,"cached_blocks":911,"cache_hit":0,"new_prefill":62840,"score_linear":11145.0,"score_lmetric":73985},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":57645,"ongoing_decode_tokens":0,"pending_prefill_tokens":57645,"num_requests":1,"active_p_offloads":0,"cached_blocks":1398,"cache_hit":0,"new_prefill":62840,"score_linear":57645.0,"score_lmetric":120485},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":138337,"ongoing_decode_tokens":0,"pending_prefill_tokens":83041,"num_requests":3,"active_p_offloads":0,"cached_blocks":655,"cache_hit":0,"new_prefill":62840,"score_linear":138337.0,"score_lmetric":437643},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":33182,"ongoing_decode_tokens":0,"pending_prefill_tokens":33182,"num_requests":1,"active_p_offloads":0,"cached_blocks":835,"cache_hit":7680,"new_prefill":55160,"score_linear":25502.0,"score_lmetric":88342},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":828,"cache_hit":59904,"new_prefill":2936,"score_linear":62064.0,"score_lmetric":2936}],"chosen_score_linear":62064.0,"chosen_score_lmetric":2936,"t_first_token":348605.288487517,"t_first_token_unix":1779779470.3024614,"t_done":348611.223318718,"t_done_unix":1779779476.2372622},{"request_id":"1324227:1:1324227:367","session_id":"1324227","input_length":14863,"t_proxy_recv":348609.767099247,"t_decision_unix":1779779474.7810404,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":-14863,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":14863,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":100019,"ongoing_decode_tokens":100019,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":795,"cache_hit":0,"new_prefill":14863,"score_linear":100019.0,"score_lmetric":29726},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":948,"cache_hit":0,"new_prefill":14863,"score_linear":62618.0,"score_lmetric":14863},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":966,"cache_hit":0,"new_prefill":14863,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":932,"cache_hit":0,"new_prefill":14863,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1527,"cache_hit":0,"new_prefill":14863,"score_linear":0.0,"score_lmetric":-14863},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":138337,"ongoing_decode_tokens":138337,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":655,"cache_hit":0,"new_prefill":14863,"score_linear":138337.0,"score_lmetric":44589},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":845,"cache_hit":0,"new_prefill":14863,"score_linear":33182.0,"score_lmetric":14863},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":184808,"ongoing_decode_tokens":184808,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":828,"cache_hit":0,"new_prefill":14863,"score_linear":184808.0,"score_lmetric":29726}],"chosen_score_linear":0.0,"chosen_score_lmetric":-14863,"t_first_token":348611.084845386,"t_first_token_unix":1779779476.0987895,"t_done":348611.336476733,"t_done_unix":1779779476.35042},{"request_id":"1279412:16:1323780:365","session_id":"1279412","input_length":75624,"t_proxy_recv":348608.076352606,"t_decision_unix":1779779473.0902944,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":72192,"affinity_cache_ratio":0.9546175817200888,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":72192,"estimated_new_tokens":3432,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":24395,"ongoing_decode_tokens":24395,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":795,"cache_hit":72192,"new_prefill":3432,"score_linear":-47797.0,"score_lmetric":3432},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":948,"cache_hit":0,"new_prefill":75624,"score_linear":62618.0,"score_lmetric":75624},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":966,"cache_hit":0,"new_prefill":75624,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":932,"cache_hit":0,"new_prefill":75624,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":65844,"ongoing_decode_tokens":54546,"pending_prefill_tokens":8226,"num_requests":1,"active_p_offloads":0,"cached_blocks":1404,"cache_hit":0,"new_prefill":75624,"score_linear":65844.0,"score_lmetric":83850},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":138337,"ongoing_decode_tokens":0,"pending_prefill_tokens":83041,"num_requests":3,"active_p_offloads":0,"cached_blocks":655,"cache_hit":0,"new_prefill":75624,"score_linear":138337.0,"score_lmetric":475995},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":46190,"ongoing_decode_tokens":46190,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":835,"cache_hit":12800,"new_prefill":62824,"score_linear":33390.0,"score_lmetric":125648},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":184808,"ongoing_decode_tokens":184808,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":828,"cache_hit":12800,"new_prefill":62824,"score_linear":172008.0,"score_lmetric":125648}],"chosen_score_linear":-47797.0,"chosen_score_lmetric":3432,"t_first_token":348609.577388523,"t_first_token_unix":1779779474.5913324,"t_done":348612.20006028,"t_done_unix":1779779477.2140036},{"request_id":"1316848:7:1325031:371","session_id":"1316848","input_length":26994,"t_proxy_recv":348612.429486114,"t_decision_unix":1779779477.4434273,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":24064,"affinity_cache_ratio":0.8914573608950137,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":24064,"estimated_new_tokens":2930,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":807,"cache_hit":24064,"new_prefill":2930,"score_linear":-24064.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":948,"cache_hit":6144,"new_prefill":20850,"score_linear":56474.0,"score_lmetric":20850},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":966,"cache_hit":0,"new_prefill":26994,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":932,"cache_hit":0,"new_prefill":26994,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":4312,"ongoing_decode_tokens":0,"pending_prefill_tokens":4312,"num_requests":0,"active_p_offloads":0,"cached_blocks":1556,"cache_hit":1024,"new_prefill":25970,"score_linear":3288.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":79663,"ongoing_decode_tokens":79663,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":661,"cache_hit":0,"new_prefill":26994,"score_linear":79663.0,"score_lmetric":26994},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":845,"cache_hit":0,"new_prefill":26994,"score_linear":33182.0,"score_lmetric":26994},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":188103,"ongoing_decode_tokens":121968,"pending_prefill_tokens":3159,"num_requests":2,"active_p_offloads":0,"cached_blocks":834,"cache_hit":0,"new_prefill":26994,"score_linear":188103.0,"score_lmetric":60306}],"chosen_score_linear":-24064.0,"chosen_score_lmetric":0,"t_first_token":348613.026029928,"t_first_token_unix":1779779478.0399733,"t_done":348613.669910471,"t_done_unix":1779779478.6838539},{"request_id":"1322540:3:1325410:373","session_id":"1322540","input_length":12118,"t_proxy_recv":348613.801176822,"t_decision_unix":1779779478.8151183,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":8704,"affinity_cache_ratio":0.7182703416405347,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":8704,"estimated_new_tokens":3414,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":812,"cache_hit":0,"new_prefill":12118,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":948,"cache_hit":0,"new_prefill":12118,"score_linear":62618.0,"score_lmetric":12118},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":966,"cache_hit":8704,"new_prefill":3414,"score_linear":-8704.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":932,"cache_hit":0,"new_prefill":12118,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":59442,"ongoing_decode_tokens":59442,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1556,"cache_hit":0,"new_prefill":12118,"score_linear":59442.0,"score_lmetric":12118},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":79663,"ongoing_decode_tokens":79663,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":661,"cache_hit":0,"new_prefill":12118,"score_linear":79663.0,"score_lmetric":12118},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":845,"cache_hit":0,"new_prefill":12118,"score_linear":33182.0,"score_lmetric":12118},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":188103,"ongoing_decode_tokens":188103,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":834,"cache_hit":0,"new_prefill":12118,"score_linear":188103.0,"score_lmetric":24236}],"chosen_score_linear":-8704.0,"chosen_score_lmetric":0,"t_first_token":348614.158549124,"t_first_token_unix":1779779479.1724982,"t_done":348614.286752262,"t_done_unix":1779779479.3006957},{"request_id":"1324964:1:1324964:370","session_id":"1324964","input_length":4312,"t_proxy_recv":348612.184896186,"t_decision_unix":1779779477.1988375,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":-4312,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":4312,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":75624,"ongoing_decode_tokens":75624,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":800,"cache_hit":0,"new_prefill":4312,"score_linear":75624.0,"score_lmetric":4312},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":948,"cache_hit":0,"new_prefill":4312,"score_linear":62618.0,"score_lmetric":4312},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":966,"cache_hit":0,"new_prefill":4312,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":932,"cache_hit":0,"new_prefill":4312,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1556,"cache_hit":0,"new_prefill":4312,"score_linear":0.0,"score_lmetric":-4312},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":79663,"ongoing_decode_tokens":79663,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":661,"cache_hit":0,"new_prefill":4312,"score_linear":79663.0,"score_lmetric":4312},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":845,"cache_hit":0,"new_prefill":4312,"score_linear":33182.0,"score_lmetric":4312},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":188103,"ongoing_decode_tokens":121968,"pending_prefill_tokens":3159,"num_requests":2,"active_p_offloads":0,"cached_blocks":834,"cache_hit":0,"new_prefill":4312,"score_linear":188103.0,"score_lmetric":14942}],"chosen_score_linear":0.0,"chosen_score_lmetric":-4312,"t_first_token":348612.453091095,"t_first_token_unix":1779779477.4670343,"t_done":348614.564060751,"t_done_unix":1779779479.5780058},{"request_id":"1317598:1:1317598:335","session_id":"1317598","input_length":79663,"t_proxy_recv":348586.258503085,"t_decision_unix":1779779451.2724445,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":79663,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":734,"cache_hit":0,"new_prefill":79663,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":947,"cache_hit":0,"new_prefill":79663,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":898,"cache_hit":0,"new_prefill":79663,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":903,"cache_hit":0,"new_prefill":79663,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":134853,"ongoing_decode_tokens":0,"pending_prefill_tokens":72901,"num_requests":2,"active_p_offloads":0,"cached_blocks":1257,"cache_hit":512,"new_prefill":79151,"score_linear":134341.0,"score_lmetric":304104},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":655,"cache_hit":0,"new_prefill":79663,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":79663,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":176419,"ongoing_decode_tokens":176419,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":810,"cache_hit":0,"new_prefill":79663,"score_linear":176419.0,"score_lmetric":159326}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348608.201013667,"t_first_token_unix":1779779473.2149577,"t_done":348616.396167786,"t_done_unix":1779779481.4101114},{"request_id":"1325987:1:1325987:376","session_id":"1325987","input_length":1823,"t_proxy_recv":348615.894697523,"t_decision_unix":1779779480.9086387,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":1823,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":812,"cache_hit":0,"new_prefill":1823,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":948,"cache_hit":0,"new_prefill":1823,"score_linear":62618.0,"score_lmetric":1823},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":66703,"ongoing_decode_tokens":0,"pending_prefill_tokens":66703,"num_requests":1,"active_p_offloads":0,"cached_blocks":972,"cache_hit":0,"new_prefill":1823,"score_linear":66703.0,"score_lmetric":68526},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":932,"cache_hit":0,"new_prefill":1823,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1565,"cache_hit":0,"new_prefill":1823,"score_linear":55130.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":79663,"ongoing_decode_tokens":79663,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":661,"cache_hit":0,"new_prefill":1823,"score_linear":79663.0,"score_lmetric":1823},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":845,"cache_hit":0,"new_prefill":1823,"score_linear":33182.0,"score_lmetric":1823},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":188103,"ongoing_decode_tokens":188103,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":834,"cache_hit":0,"new_prefill":1823,"score_linear":188103.0,"score_lmetric":3646}],"chosen_score_linear":55130.0,"chosen_score_lmetric":0,"t_first_token":348616.104775626,"t_first_token_unix":1779779481.1187189,"t_done":348616.448433656,"t_done_unix":1779779481.4623775},{"request_id":"1326192:1:1326192:378","session_id":"1326192","input_length":9513,"t_proxy_recv":348616.774138154,"t_decision_unix":1779779481.7880793,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":9513,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":81008,"ongoing_decode_tokens":0,"pending_prefill_tokens":5232,"num_requests":1,"active_p_offloads":0,"cached_blocks":812,"cache_hit":0,"new_prefill":9513,"score_linear":81008.0,"score_lmetric":14745},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":948,"cache_hit":0,"new_prefill":9513,"score_linear":62618.0,"score_lmetric":9513},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":66703,"ongoing_decode_tokens":0,"pending_prefill_tokens":66703,"num_requests":1,"active_p_offloads":0,"cached_blocks":972,"cache_hit":0,"new_prefill":9513,"score_linear":66703.0,"score_lmetric":76216},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":932,"cache_hit":0,"new_prefill":9513,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1568,"cache_hit":0,"new_prefill":9513,"score_linear":55130.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":81220,"ongoing_decode_tokens":0,"pending_prefill_tokens":836,"num_requests":1,"active_p_offloads":0,"cached_blocks":818,"cache_hit":0,"new_prefill":9513,"score_linear":81220.0,"score_lmetric":10349},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":845,"cache_hit":0,"new_prefill":9513,"score_linear":33182.0,"score_lmetric":9513},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":188103,"ongoing_decode_tokens":188103,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":834,"cache_hit":0,"new_prefill":9513,"score_linear":188103.0,"score_lmetric":19026}],"chosen_score_linear":55130.0,"chosen_score_lmetric":0,"t_first_token":348617.530382479,"t_first_token_unix":1779779482.5443258,"t_done":348618.475423779,"t_done_unix":1779779483.4893675},{"request_id":"1317598:2:1324364:368","session_id":"1317598","input_length":81220,"t_proxy_recv":348616.410629645,"t_decision_unix":1779779481.424571,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":80384,"affinity_cache_ratio":0.9897069687269145,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":80384,"estimated_new_tokens":836,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":812,"cache_hit":0,"new_prefill":81220,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":948,"cache_hit":0,"new_prefill":81220,"score_linear":62618.0,"score_lmetric":81220},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":66703,"ongoing_decode_tokens":0,"pending_prefill_tokens":66703,"num_requests":1,"active_p_offloads":0,"cached_blocks":972,"cache_hit":0,"new_prefill":81220,"score_linear":66703.0,"score_lmetric":147923},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":932,"cache_hit":0,"new_prefill":81220,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":56953,"ongoing_decode_tokens":56953,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1565,"cache_hit":512,"new_prefill":80708,"score_linear":56441.0,"score_lmetric":80708},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":818,"cache_hit":80384,"new_prefill":836,"score_linear":-80384.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":845,"cache_hit":0,"new_prefill":81220,"score_linear":33182.0,"score_lmetric":81220},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":188103,"ongoing_decode_tokens":188103,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":834,"cache_hit":0,"new_prefill":81220,"score_linear":188103.0,"score_lmetric":162440}],"chosen_score_linear":-80384.0,"chosen_score_lmetric":0,"t_first_token":348616.977684017,"t_first_token_unix":1779779481.9916277,"t_done":348619.837352172,"t_done_unix":1779779484.8512983},{"request_id":"1269373:13:1317516:334","session_id":"1269373","input_length":66135,"t_proxy_recv":348611.235273393,"t_decision_unix":1779779476.2492146,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":62976,"affinity_cache_ratio":0.9522340666817872,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":62976,"estimated_new_tokens":3159,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":75624,"ongoing_decode_tokens":75624,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":800,"cache_hit":7680,"new_prefill":58455,"score_linear":67944.0,"score_lmetric":58455},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":948,"cache_hit":0,"new_prefill":66135,"score_linear":62618.0,"score_lmetric":66135},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":966,"cache_hit":0,"new_prefill":66135,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":932,"cache_hit":0,"new_prefill":66135,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":14863,"ongoing_decode_tokens":14863,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1527,"cache_hit":7680,"new_prefill":58455,"score_linear":7183.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":79663,"ongoing_decode_tokens":79663,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":661,"cache_hit":0,"new_prefill":66135,"score_linear":79663.0,"score_lmetric":66135},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":845,"cache_hit":7680,"new_prefill":58455,"score_linear":25502.0,"score_lmetric":58455},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":834,"cache_hit":62976,"new_prefill":3159,"score_linear":58992.0,"score_lmetric":3159}],"chosen_score_linear":58992.0,"chosen_score_lmetric":3159,"t_first_token":348612.567789736,"t_first_token_unix":1779779477.5817337,"t_done":348619.986687162,"t_done_unix":1779779485.000631},{"request_id":"1326192:2:1327234:381","session_id":"1326192","input_length":15253,"t_proxy_recv":348620.335338724,"t_decision_unix":1779779485.34928,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":9216,"affinity_cache_ratio":0.6042090080639874,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":9216,"estimated_new_tokens":6037,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":109128,"ongoing_decode_tokens":109128,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":812,"cache_hit":0,"new_prefill":15253,"score_linear":109128.0,"score_lmetric":30506},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":948,"cache_hit":0,"new_prefill":15253,"score_linear":62618.0,"score_lmetric":15253},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":79952,"ongoing_decode_tokens":0,"pending_prefill_tokens":68176,"num_requests":2,"active_p_offloads":0,"cached_blocks":972,"cache_hit":0,"new_prefill":15253,"score_linear":79952.0,"score_lmetric":166858},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":932,"cache_hit":0,"new_prefill":15253,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1586,"cache_hit":9216,"new_prefill":6037,"score_linear":45914.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":15253,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":845,"cache_hit":0,"new_prefill":15253,"score_linear":33182.0,"score_lmetric":15253},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":188808,"ongoing_decode_tokens":188808,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":841,"cache_hit":0,"new_prefill":15253,"score_linear":188808.0,"score_lmetric":30506}],"chosen_score_linear":45914.0,"chosen_score_lmetric":0,"t_first_token":348621.033602563,"t_first_token_unix":1779779486.0475547,"t_done":348621.910921729,"t_done_unix":1779779486.924865},{"request_id":"1324964:2:1327382:382","session_id":"1324964","input_length":5146,"t_proxy_recv":348620.938016246,"t_decision_unix":1779779485.9519572,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":4608,"affinity_cache_ratio":0.895452778857365,"affinity_num_requests":1,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":4608,"estimated_new_tokens":538,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":109128,"ongoing_decode_tokens":109128,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":812,"cache_hit":0,"new_prefill":5146,"score_linear":109128.0,"score_lmetric":10292},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":948,"cache_hit":0,"new_prefill":5146,"score_linear":62618.0,"score_lmetric":5146},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":79952,"ongoing_decode_tokens":0,"pending_prefill_tokens":68176,"num_requests":2,"active_p_offloads":0,"cached_blocks":972,"cache_hit":0,"new_prefill":5146,"score_linear":79952.0,"score_lmetric":146644},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":932,"cache_hit":0,"new_prefill":5146,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":70383,"ongoing_decode_tokens":55130,"pending_prefill_tokens":6037,"num_requests":1,"active_p_offloads":0,"cached_blocks":1586,"cache_hit":4608,"new_prefill":538,"score_linear":65775.0,"score_lmetric":6575},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":5146,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":845,"cache_hit":0,"new_prefill":5146,"score_linear":33182.0,"score_lmetric":5146},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":188808,"ongoing_decode_tokens":188808,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":841,"cache_hit":0,"new_prefill":5146,"score_linear":188808.0,"score_lmetric":10292}],"chosen_score_linear":65775.0,"chosen_score_lmetric":6575,"t_first_token":348621.181729204,"t_first_token_unix":1779779486.195673,"t_done":348622.104654872,"t_done_unix":1779779487.1185982},{"request_id":"1316848:8:1326380:379","session_id":"1316848","input_length":28120,"t_proxy_recv":348617.386028861,"t_decision_unix":1779779482.39997,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":26624,"affinity_cache_ratio":0.9467994310099573,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":26624,"estimated_new_tokens":1496,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":81008,"ongoing_decode_tokens":0,"pending_prefill_tokens":5232,"num_requests":1,"active_p_offloads":0,"cached_blocks":812,"cache_hit":26624,"new_prefill":1496,"score_linear":54384.0,"score_lmetric":6728},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":948,"cache_hit":6144,"new_prefill":21976,"score_linear":56474.0,"score_lmetric":21976},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":66703,"ongoing_decode_tokens":0,"pending_prefill_tokens":66703,"num_requests":1,"active_p_offloads":0,"cached_blocks":972,"cache_hit":0,"new_prefill":28120,"score_linear":66703.0,"score_lmetric":94823},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":932,"cache_hit":0,"new_prefill":28120,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":64643,"ongoing_decode_tokens":55130,"pending_prefill_tokens":9513,"num_requests":1,"active_p_offloads":0,"cached_blocks":1568,"cache_hit":1024,"new_prefill":27096,"score_linear":63619.0,"score_lmetric":36609},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":81220,"ongoing_decode_tokens":81220,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":818,"cache_hit":0,"new_prefill":28120,"score_linear":81220.0,"score_lmetric":28120},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":845,"cache_hit":0,"new_prefill":28120,"score_linear":33182.0,"score_lmetric":28120},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":188103,"ongoing_decode_tokens":188103,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":834,"cache_hit":0,"new_prefill":28120,"score_linear":188103.0,"score_lmetric":56240}],"chosen_score_linear":54384.0,"chosen_score_lmetric":6728,"t_first_token":348619.465851886,"t_first_token_unix":1779779484.4797955,"t_done":348622.955598543,"t_done_unix":1779779487.9695418},{"request_id":"1269373:14:1321512:354","session_id":"1269373","input_length":66840,"t_proxy_recv":348619.998843093,"t_decision_unix":1779779485.0127847,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":66560,"affinity_cache_ratio":0.9958108916816277,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":66560,"estimated_new_tokens":280,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":109128,"ongoing_decode_tokens":109128,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":812,"cache_hit":7680,"new_prefill":59160,"score_linear":101448.0,"score_lmetric":118320},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":948,"cache_hit":0,"new_prefill":66840,"score_linear":62618.0,"score_lmetric":66840},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":79952,"ongoing_decode_tokens":0,"pending_prefill_tokens":68176,"num_requests":2,"active_p_offloads":0,"cached_blocks":972,"cache_hit":0,"new_prefill":66840,"score_linear":79952.0,"score_lmetric":270032},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":932,"cache_hit":0,"new_prefill":66840,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1586,"cache_hit":7680,"new_prefill":59160,"score_linear":47450.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":66840,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":845,"cache_hit":7680,"new_prefill":59160,"score_linear":25502.0,"score_lmetric":59160},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":841,"cache_hit":66560,"new_prefill":280,"score_linear":55408.0,"score_lmetric":280}],"chosen_score_linear":55408.0,"chosen_score_lmetric":280,"t_first_token":348620.22037289,"t_first_token_unix":1779779485.2343166,"t_done":348624.22169442,"t_done_unix":1779779489.235638},{"request_id":"1324964:3:1328499:384","session_id":"1324964","input_length":5234,"t_proxy_recv":348624.953858989,"t_decision_unix":1779779489.9678004,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":5120,"affinity_cache_ratio":0.9782193351165457,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":5120,"estimated_new_tokens":114,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":81008,"ongoing_decode_tokens":81008,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":815,"cache_hit":0,"new_prefill":5234,"score_linear":81008.0,"score_lmetric":5234},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":948,"cache_hit":0,"new_prefill":5234,"score_linear":62618.0,"score_lmetric":5234},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":79952,"ongoing_decode_tokens":0,"pending_prefill_tokens":68176,"num_requests":2,"active_p_offloads":0,"cached_blocks":972,"cache_hit":0,"new_prefill":5234,"score_linear":79952.0,"score_lmetric":146820},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":932,"cache_hit":0,"new_prefill":5234,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1598,"cache_hit":5120,"new_prefill":114,"score_linear":50010.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":5234,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":845,"cache_hit":0,"new_prefill":5234,"score_linear":33182.0,"score_lmetric":5234},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":191009,"ongoing_decode_tokens":121968,"pending_prefill_tokens":1969,"num_requests":2,"active_p_offloads":0,"cached_blocks":842,"cache_hit":0,"new_prefill":5234,"score_linear":191009.0,"score_lmetric":14406}],"chosen_score_linear":50010.0,"chosen_score_lmetric":0,"t_first_token":348624.981287402,"t_first_token_unix":1779779489.9952304,"t_done":348625.37158306,"t_done_unix":1779779490.3855262},{"request_id":"1317598:3:1328678:385","session_id":"1317598","input_length":81827,"t_proxy_recv":348625.632474417,"t_decision_unix":1779779490.646415,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":81408,"affinity_cache_ratio":0.9948794407713836,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":81408,"estimated_new_tokens":419,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":81008,"ongoing_decode_tokens":81008,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":815,"cache_hit":0,"new_prefill":81827,"score_linear":81008.0,"score_lmetric":81827},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":948,"cache_hit":0,"new_prefill":81827,"score_linear":62618.0,"score_lmetric":81827},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":79952,"ongoing_decode_tokens":0,"pending_prefill_tokens":68176,"num_requests":2,"active_p_offloads":0,"cached_blocks":972,"cache_hit":0,"new_prefill":81827,"score_linear":79952.0,"score_lmetric":300006},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":932,"cache_hit":0,"new_prefill":81827,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1598,"cache_hit":512,"new_prefill":81315,"score_linear":54618.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":820,"cache_hit":81408,"new_prefill":419,"score_linear":-81408.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":845,"cache_hit":0,"new_prefill":81827,"score_linear":33182.0,"score_lmetric":81827},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":191009,"ongoing_decode_tokens":191009,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":842,"cache_hit":0,"new_prefill":81827,"score_linear":191009.0,"score_lmetric":163654}],"chosen_score_linear":-81408.0,"chosen_score_lmetric":0,"t_first_token":348626.044489345,"t_first_token_unix":1779779491.0584328,"t_done":348627.544778158,"t_done_unix":1779779492.558722},{"request_id":"1322225:1:1322225:358","session_id":"1322225","input_length":33182,"t_proxy_recv":348602.733131557,"t_decision_unix":1779779467.747073,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":33182,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":93542,"ongoing_decode_tokens":93542,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":752,"cache_hit":0,"new_prefill":33182,"score_linear":93542.0,"score_lmetric":66364},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":948,"cache_hit":0,"new_prefill":33182,"score_linear":62618.0,"score_lmetric":33182},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":949,"cache_hit":0,"new_prefill":33182,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":911,"cache_hit":0,"new_prefill":33182,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":57645,"ongoing_decode_tokens":0,"pending_prefill_tokens":57645,"num_requests":1,"active_p_offloads":0,"cached_blocks":1398,"cache_hit":0,"new_prefill":33182,"score_linear":57645.0,"score_lmetric":90827},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":138337,"ongoing_decode_tokens":0,"pending_prefill_tokens":83041,"num_requests":3,"active_p_offloads":0,"cached_blocks":655,"cache_hit":0,"new_prefill":33182,"score_linear":138337.0,"score_lmetric":348669},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":835,"cache_hit":0,"new_prefill":33182,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":181414,"ongoing_decode_tokens":181414,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":826,"cache_hit":0,"new_prefill":33182,"score_linear":181414.0,"score_lmetric":66364}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348607.440753177,"t_first_token_unix":1779779472.4546967,"t_done":348628.030092302,"t_done_unix":1779779493.0440361},{"request_id":"1329110:1:1329110:389","session_id":"1329110","input_length":9292,"t_proxy_recv":348627.166908575,"t_decision_unix":1779779492.1808496,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":9292,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":109623,"ongoing_decode_tokens":109623,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":815,"cache_hit":0,"new_prefill":9292,"score_linear":109623.0,"score_lmetric":18584},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":96882,"ongoing_decode_tokens":62618,"pending_prefill_tokens":7640,"num_requests":2,"active_p_offloads":0,"cached_blocks":948,"cache_hit":0,"new_prefill":9292,"score_linear":96882.0,"score_lmetric":33864},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":79952,"ongoing_decode_tokens":0,"pending_prefill_tokens":68176,"num_requests":2,"active_p_offloads":0,"cached_blocks":972,"cache_hit":0,"new_prefill":9292,"score_linear":79952.0,"score_lmetric":154936},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":36880,"ongoing_decode_tokens":0,"pending_prefill_tokens":36880,"num_requests":1,"active_p_offloads":0,"cached_blocks":932,"cache_hit":0,"new_prefill":9292,"score_linear":36880.0,"score_lmetric":46172},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1598,"cache_hit":0,"new_prefill":9292,"score_linear":55130.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":81827,"ongoing_decode_tokens":81827,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":9292,"score_linear":81827.0,"score_lmetric":9292},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":845,"cache_hit":0,"new_prefill":9292,"score_linear":33182.0,"score_lmetric":9292},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":191009,"ongoing_decode_tokens":191009,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":842,"cache_hit":0,"new_prefill":9292,"score_linear":191009.0,"score_lmetric":18584}],"chosen_score_linear":55130.0,"chosen_score_lmetric":0,"t_first_token":348627.918164005,"t_first_token_unix":1779779492.9321077,"t_done":348628.564046487,"t_done_unix":1779779493.5779896},{"request_id":"1325987:2:1329336:391","session_id":"1325987","input_length":7423,"t_proxy_recv":348628.016976129,"t_decision_unix":1779779493.0309176,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":4,"chosen_idx":5,"affinity_cache_hit":1536,"affinity_cache_ratio":0.20692442408729625,"affinity_num_requests":2,"avg_num_requests":1.5,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":7423,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":109623,"ongoing_decode_tokens":109623,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":815,"cache_hit":0,"new_prefill":7423,"score_linear":109623.0,"score_lmetric":14846},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":96882,"ongoing_decode_tokens":62618,"pending_prefill_tokens":7640,"num_requests":2,"active_p_offloads":0,"cached_blocks":948,"cache_hit":0,"new_prefill":7423,"score_linear":96882.0,"score_lmetric":30126},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":79952,"ongoing_decode_tokens":0,"pending_prefill_tokens":68176,"num_requests":2,"active_p_offloads":0,"cached_blocks":972,"cache_hit":0,"new_prefill":7423,"score_linear":79952.0,"score_lmetric":151198},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":36880,"ongoing_decode_tokens":0,"pending_prefill_tokens":36880,"num_requests":1,"active_p_offloads":0,"cached_blocks":932,"cache_hit":0,"new_prefill":7423,"score_linear":36880.0,"score_lmetric":44303},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":69836,"ongoing_decode_tokens":69836,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1598,"cache_hit":1536,"new_prefill":5887,"score_linear":68300.0,"score_lmetric":11774},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":821,"cache_hit":0,"new_prefill":7423,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":845,"cache_hit":0,"new_prefill":7423,"score_linear":33182.0,"score_lmetric":7423},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":191009,"ongoing_decode_tokens":191009,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":842,"cache_hit":0,"new_prefill":7423,"score_linear":191009.0,"score_lmetric":14846}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348628.520244106,"t_first_token_unix":1779779493.5341876,"t_done":348628.725135754,"t_done_unix":1779779493.7390788},{"request_id":"1324964:4:1329201:390","session_id":"1324964","input_length":5414,"t_proxy_recv":348627.479892724,"t_decision_unix":1779779492.4938335,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":5120,"affinity_cache_ratio":0.9456963428149243,"affinity_num_requests":1,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":5120,"estimated_new_tokens":294,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":109623,"ongoing_decode_tokens":109623,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":815,"cache_hit":0,"new_prefill":5414,"score_linear":109623.0,"score_lmetric":10828},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":96882,"ongoing_decode_tokens":62618,"pending_prefill_tokens":7640,"num_requests":2,"active_p_offloads":0,"cached_blocks":948,"cache_hit":0,"new_prefill":5414,"score_linear":96882.0,"score_lmetric":26108},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":79952,"ongoing_decode_tokens":0,"pending_prefill_tokens":68176,"num_requests":2,"active_p_offloads":0,"cached_blocks":972,"cache_hit":0,"new_prefill":5414,"score_linear":79952.0,"score_lmetric":147180},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":36880,"ongoing_decode_tokens":0,"pending_prefill_tokens":36880,"num_requests":1,"active_p_offloads":0,"cached_blocks":932,"cache_hit":0,"new_prefill":5414,"score_linear":36880.0,"score_lmetric":42294},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":64422,"ongoing_decode_tokens":55130,"pending_prefill_tokens":9292,"num_requests":1,"active_p_offloads":0,"cached_blocks":1598,"cache_hit":5120,"new_prefill":294,"score_linear":59302.0,"score_lmetric":9586},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":81827,"ongoing_decode_tokens":81827,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":5414,"score_linear":81827.0,"score_lmetric":5414},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":845,"cache_hit":0,"new_prefill":5414,"score_linear":33182.0,"score_lmetric":5414},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":191009,"ongoing_decode_tokens":191009,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":842,"cache_hit":0,"new_prefill":5414,"score_linear":191009.0,"score_lmetric":10828}],"chosen_score_linear":59302.0,"chosen_score_lmetric":9586,"t_first_token":348627.952311074,"t_first_token_unix":1779779492.9662547,"t_done":348628.73426768,"t_done_unix":1779779493.748211},{"request_id":"1316848:9:1328693:386","session_id":"1316848","input_length":28615,"t_proxy_recv":348625.669206746,"t_decision_unix":1779779490.6831484,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":28160,"affinity_cache_ratio":0.9840992486458151,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":28160,"estimated_new_tokens":455,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":81008,"ongoing_decode_tokens":81008,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":815,"cache_hit":28160,"new_prefill":455,"score_linear":52848.0,"score_lmetric":455},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":948,"cache_hit":6144,"new_prefill":22471,"score_linear":56474.0,"score_lmetric":22471},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":79952,"ongoing_decode_tokens":0,"pending_prefill_tokens":68176,"num_requests":2,"active_p_offloads":0,"cached_blocks":972,"cache_hit":0,"new_prefill":28615,"score_linear":79952.0,"score_lmetric":193582},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":932,"cache_hit":0,"new_prefill":28615,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1598,"cache_hit":1024,"new_prefill":27591,"score_linear":54106.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":81827,"ongoing_decode_tokens":0,"pending_prefill_tokens":419,"num_requests":1,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":28615,"score_linear":81827.0,"score_lmetric":29034},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":845,"cache_hit":0,"new_prefill":28615,"score_linear":33182.0,"score_lmetric":28615},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":191009,"ongoing_decode_tokens":191009,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":842,"cache_hit":0,"new_prefill":28615,"score_linear":191009.0,"score_lmetric":57230}],"chosen_score_linear":52848.0,"chosen_score_lmetric":455,"t_first_token":348625.79226158,"t_first_token_unix":1779779490.8062053,"t_done":348628.830084343,"t_done_unix":1779779493.8440273},{"request_id":"1279412:17:1326166:377","session_id":"1279412","input_length":81008,"t_proxy_recv":348616.621505948,"t_decision_unix":1779779481.635447,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":75776,"affinity_cache_ratio":0.9354137862927119,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":75776,"estimated_new_tokens":5232,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":812,"cache_hit":75776,"new_prefill":5232,"score_linear":-75776.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":948,"cache_hit":0,"new_prefill":81008,"score_linear":62618.0,"score_lmetric":81008},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":66703,"ongoing_decode_tokens":0,"pending_prefill_tokens":66703,"num_requests":1,"active_p_offloads":0,"cached_blocks":972,"cache_hit":0,"new_prefill":81008,"score_linear":66703.0,"score_lmetric":147711},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":932,"cache_hit":0,"new_prefill":81008,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1568,"cache_hit":0,"new_prefill":81008,"score_linear":55130.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":81220,"ongoing_decode_tokens":0,"pending_prefill_tokens":836,"num_requests":1,"active_p_offloads":0,"cached_blocks":818,"cache_hit":0,"new_prefill":81008,"score_linear":81220.0,"score_lmetric":81844},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":845,"cache_hit":12800,"new_prefill":68208,"score_linear":20382.0,"score_lmetric":68208},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":188103,"ongoing_decode_tokens":188103,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":834,"cache_hit":12800,"new_prefill":68208,"score_linear":175303.0,"score_lmetric":136416}],"chosen_score_linear":-75776.0,"chosen_score_lmetric":0,"t_first_token":348619.235580719,"t_first_token_unix":1779779484.2495246,"t_done":348629.408385439,"t_done_unix":1779779494.4223292},{"request_id":"1322540:4:1326861:380","session_id":"1322540","input_length":13249,"t_proxy_recv":348619.050996179,"t_decision_unix":1779779484.064937,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":11776,"affinity_cache_ratio":0.8888217978715375,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":11776,"estimated_new_tokens":1473,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":109128,"ongoing_decode_tokens":0,"pending_prefill_tokens":6728,"num_requests":2,"active_p_offloads":0,"cached_blocks":812,"cache_hit":0,"new_prefill":13249,"score_linear":109128.0,"score_lmetric":39954},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":948,"cache_hit":0,"new_prefill":13249,"score_linear":62618.0,"score_lmetric":13249},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":66703,"ongoing_decode_tokens":0,"pending_prefill_tokens":66703,"num_requests":1,"active_p_offloads":0,"cached_blocks":972,"cache_hit":11776,"new_prefill":1473,"score_linear":54927.0,"score_lmetric":68176},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":932,"cache_hit":0,"new_prefill":13249,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1586,"cache_hit":0,"new_prefill":13249,"score_linear":55130.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":81220,"ongoing_decode_tokens":81220,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":818,"cache_hit":0,"new_prefill":13249,"score_linear":81220.0,"score_lmetric":13249},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":845,"cache_hit":0,"new_prefill":13249,"score_linear":33182.0,"score_lmetric":13249},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":188103,"ongoing_decode_tokens":188103,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":834,"cache_hit":0,"new_prefill":13249,"score_linear":188103.0,"score_lmetric":26498}],"chosen_score_linear":54927.0,"chosen_score_lmetric":68176,"t_first_token":348630.116440178,"t_first_token_unix":1779779495.1303937,"t_done":348630.319704267,"t_done_unix":1779779495.3336477},{"request_id":"1325541:1:1325541:374","session_id":"1325541","input_length":66703,"t_proxy_recv":348614.390045734,"t_decision_unix":1779779479.4039872,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":66703,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":812,"cache_hit":0,"new_prefill":66703,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":948,"cache_hit":0,"new_prefill":66703,"score_linear":62618.0,"score_lmetric":66703},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":972,"cache_hit":0,"new_prefill":66703,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":932,"cache_hit":0,"new_prefill":66703,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":59442,"ongoing_decode_tokens":59442,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1556,"cache_hit":0,"new_prefill":66703,"score_linear":59442.0,"score_lmetric":66703},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":79663,"ongoing_decode_tokens":79663,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":661,"cache_hit":0,"new_prefill":66703,"score_linear":79663.0,"score_lmetric":66703},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":845,"cache_hit":0,"new_prefill":66703,"score_linear":33182.0,"score_lmetric":66703},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":188103,"ongoing_decode_tokens":188103,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":834,"cache_hit":0,"new_prefill":66703,"score_linear":188103.0,"score_lmetric":133406}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348630.115117472,"t_first_token_unix":1779779495.1290627,"t_done":348630.41662336,"t_done_unix":1779779495.4305665},{"request_id":"1329470:1:1329470:392","session_id":"1329470","input_length":10847,"t_proxy_recv":348628.501879566,"t_decision_unix":1779779493.5158207,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.5,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":10847,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":109623,"ongoing_decode_tokens":109623,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":815,"cache_hit":0,"new_prefill":10847,"score_linear":109623.0,"score_lmetric":21694},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":96882,"ongoing_decode_tokens":62618,"pending_prefill_tokens":7640,"num_requests":2,"active_p_offloads":0,"cached_blocks":948,"cache_hit":0,"new_prefill":10847,"score_linear":96882.0,"score_lmetric":36974},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":79952,"ongoing_decode_tokens":0,"pending_prefill_tokens":68176,"num_requests":2,"active_p_offloads":0,"cached_blocks":972,"cache_hit":0,"new_prefill":10847,"score_linear":79952.0,"score_lmetric":158046},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":36880,"ongoing_decode_tokens":0,"pending_prefill_tokens":36880,"num_requests":1,"active_p_offloads":0,"cached_blocks":932,"cache_hit":0,"new_prefill":10847,"score_linear":36880.0,"score_lmetric":47727},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":69836,"ongoing_decode_tokens":69836,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1598,"cache_hit":0,"new_prefill":10847,"score_linear":69836.0,"score_lmetric":21694},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":7423,"ongoing_decode_tokens":0,"pending_prefill_tokens":7423,"num_requests":1,"active_p_offloads":0,"cached_blocks":821,"cache_hit":0,"new_prefill":10847,"score_linear":7423.0,"score_lmetric":18270},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":911,"cache_hit":0,"new_prefill":10847,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":191009,"ongoing_decode_tokens":191009,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":842,"cache_hit":0,"new_prefill":10847,"score_linear":191009.0,"score_lmetric":21694}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348629.383525234,"t_first_token_unix":1779779494.3974686,"t_done":348630.887420701,"t_done_unix":1779779495.9013636},{"request_id":"1328753:1:1328753:387","session_id":"1328753","input_length":36880,"t_proxy_recv":348625.902204584,"t_decision_unix":1779779490.9161453,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":36880,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":109623,"ongoing_decode_tokens":109623,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":815,"cache_hit":0,"new_prefill":36880,"score_linear":109623.0,"score_lmetric":73760},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":948,"cache_hit":0,"new_prefill":36880,"score_linear":62618.0,"score_lmetric":36880},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":79952,"ongoing_decode_tokens":0,"pending_prefill_tokens":68176,"num_requests":2,"active_p_offloads":0,"cached_blocks":972,"cache_hit":0,"new_prefill":36880,"score_linear":79952.0,"score_lmetric":210112},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":932,"cache_hit":0,"new_prefill":36880,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1598,"cache_hit":0,"new_prefill":36880,"score_linear":55130.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":81827,"ongoing_decode_tokens":0,"pending_prefill_tokens":419,"num_requests":1,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":36880,"score_linear":81827.0,"score_lmetric":37299},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":845,"cache_hit":0,"new_prefill":36880,"score_linear":33182.0,"score_lmetric":36880},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":191009,"ongoing_decode_tokens":191009,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":842,"cache_hit":0,"new_prefill":36880,"score_linear":191009.0,"score_lmetric":73760}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348631.438767514,"t_first_token_unix":1779779496.452711,"t_done":348631.826831607,"t_done_unix":1779779496.8407748},{"request_id":"1330130:1:1330130:396","session_id":"1330130","input_length":11725,"t_proxy_recv":348630.770039054,"t_decision_unix":1779779495.7839804,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":11725,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":827,"cache_hit":0,"new_prefill":11725,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":96882,"ongoing_decode_tokens":96882,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":948,"cache_hit":0,"new_prefill":11725,"score_linear":96882.0,"score_lmetric":23450},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1104,"cache_hit":0,"new_prefill":11725,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":36880,"ongoing_decode_tokens":0,"pending_prefill_tokens":36880,"num_requests":1,"active_p_offloads":0,"cached_blocks":932,"cache_hit":0,"new_prefill":11725,"score_linear":36880.0,"score_lmetric":48605},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":145735,"ongoing_decode_tokens":55130,"pending_prefill_tokens":493,"num_requests":1,"active_p_offloads":0,"cached_blocks":1616,"cache_hit":0,"new_prefill":11725,"score_linear":145735.0,"score_lmetric":12218},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":835,"cache_hit":0,"new_prefill":11725,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":10847,"ongoing_decode_tokens":10847,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":911,"cache_hit":0,"new_prefill":11725,"score_linear":10847.0,"score_lmetric":11725},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":191009,"ongoing_decode_tokens":191009,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":842,"cache_hit":0,"new_prefill":11725,"score_linear":191009.0,"score_lmetric":23450}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348631.714101735,"t_first_token_unix":1779779496.7280452,"t_done":348632.17168088,"t_done_unix":1779779497.185624},{"request_id":"1328753:2:1330033:395","session_id":"1328753","input_length":37007,"t_proxy_recv":348631.83490833,"t_decision_unix":1779779496.84885,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":36864,"affinity_cache_ratio":0.9961358661874781,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":36864,"estimated_new_tokens":143,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":827,"cache_hit":0,"new_prefill":37007,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":96882,"ongoing_decode_tokens":96882,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":948,"cache_hit":0,"new_prefill":37007,"score_linear":96882.0,"score_lmetric":74014},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1104,"cache_hit":0,"new_prefill":37007,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":36864,"new_prefill":143,"score_linear":-36864.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":152426,"ongoing_decode_tokens":55130,"pending_prefill_tokens":2064,"num_requests":2,"active_p_offloads":0,"cached_blocks":1616,"cache_hit":0,"new_prefill":37007,"score_linear":152426.0,"score_lmetric":78142},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":11725,"ongoing_decode_tokens":11725,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":835,"cache_hit":0,"new_prefill":37007,"score_linear":11725.0,"score_lmetric":37007},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":932,"cache_hit":0,"new_prefill":37007,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":191009,"ongoing_decode_tokens":191009,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":842,"cache_hit":0,"new_prefill":37007,"score_linear":191009.0,"score_lmetric":74014}],"chosen_score_linear":-36864.0,"chosen_score_lmetric":0,"t_first_token":348631.93165099,"t_first_token_unix":1779779496.9455948,"t_done":348632.389109247,"t_done_unix":1779779497.4030526},{"request_id":"1268861:5:1329060:388","session_id":"1268861","input_length":34264,"t_proxy_recv":348627.044351307,"t_decision_unix":1779779492.0582924,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":26624,"affinity_cache_ratio":0.7770254494513191,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":26624,"estimated_new_tokens":7640,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":109623,"ongoing_decode_tokens":109623,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":815,"cache_hit":0,"new_prefill":34264,"score_linear":109623.0,"score_lmetric":68528},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":948,"cache_hit":26624,"new_prefill":7640,"score_linear":35994.0,"score_lmetric":7640},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":79952,"ongoing_decode_tokens":0,"pending_prefill_tokens":68176,"num_requests":2,"active_p_offloads":0,"cached_blocks":972,"cache_hit":0,"new_prefill":34264,"score_linear":79952.0,"score_lmetric":204880},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":36880,"ongoing_decode_tokens":0,"pending_prefill_tokens":36880,"num_requests":1,"active_p_offloads":0,"cached_blocks":932,"cache_hit":0,"new_prefill":34264,"score_linear":36880.0,"score_lmetric":71144},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1598,"cache_hit":0,"new_prefill":34264,"score_linear":55130.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":81827,"ongoing_decode_tokens":81827,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":34264,"score_linear":81827.0,"score_lmetric":34264},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":845,"cache_hit":0,"new_prefill":34264,"score_linear":33182.0,"score_lmetric":34264},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":191009,"ongoing_decode_tokens":191009,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":842,"cache_hit":12288,"new_prefill":21976,"score_linear":178721.0,"score_lmetric":43952}],"chosen_score_linear":35994.0,"chosen_score_lmetric":7640,"t_first_token":348628.67074386,"t_first_token_unix":1779779493.6846874,"t_done":348633.102252582,"t_done_unix":1779779498.1161962},{"request_id":"1269373:15:1325630:375","session_id":"1269373","input_length":69041,"t_proxy_recv":348624.236430566,"t_decision_unix":1779779489.2503717,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":67072,"affinity_cache_ratio":0.971480714358135,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":67072,"estimated_new_tokens":1969,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":81008,"ongoing_decode_tokens":81008,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":815,"cache_hit":7680,"new_prefill":61361,"score_linear":73328.0,"score_lmetric":61361},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":948,"cache_hit":0,"new_prefill":69041,"score_linear":62618.0,"score_lmetric":69041},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":79952,"ongoing_decode_tokens":0,"pending_prefill_tokens":68176,"num_requests":2,"active_p_offloads":0,"cached_blocks":972,"cache_hit":0,"new_prefill":69041,"score_linear":79952.0,"score_lmetric":274434},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":932,"cache_hit":0,"new_prefill":69041,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1598,"cache_hit":7680,"new_prefill":61361,"score_linear":47450.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":69041,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":845,"cache_hit":7680,"new_prefill":61361,"score_linear":25502.0,"score_lmetric":61361},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":842,"cache_hit":67072,"new_prefill":1969,"score_linear":54896.0,"score_lmetric":1969}],"chosen_score_linear":54896.0,"chosen_score_lmetric":1969,"t_first_token":348625.258894375,"t_first_token_unix":1779779490.272838,"t_done":348634.071177252,"t_done_unix":1779779499.085121},{"request_id":"1330881:1:1330881:400","session_id":"1330881","input_length":3866,"t_proxy_recv":348633.528542869,"t_decision_unix":1779779498.542484,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":3866,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":28977,"ongoing_decode_tokens":28977,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":827,"cache_hit":0,"new_prefill":3866,"score_linear":28977.0,"score_lmetric":3866},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":0,"new_prefill":3866,"score_linear":62618.0,"score_lmetric":3866},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1104,"cache_hit":0,"new_prefill":3866,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":3866,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":178114,"ongoing_decode_tokens":55130,"pending_prefill_tokens":12904,"num_requests":3,"active_p_offloads":0,"cached_blocks":1616,"cache_hit":0,"new_prefill":3866,"score_linear":178114.0,"score_lmetric":50310},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":858,"cache_hit":0,"new_prefill":3866,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":932,"cache_hit":0,"new_prefill":3866,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":191009,"ongoing_decode_tokens":191009,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":842,"cache_hit":0,"new_prefill":3866,"score_linear":191009.0,"score_lmetric":7732}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348633.770016382,"t_first_token_unix":1779779498.7839606,"t_done":348634.134591325,"t_done_unix":1779779499.1485345},{"request_id":"1316848:10:1330509:398","session_id":"1316848","input_length":28977,"t_proxy_recv":348632.140858663,"t_decision_unix":1779779497.1548,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":28672,"affinity_cache_ratio":0.9894744107395521,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":28672,"estimated_new_tokens":305,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":827,"cache_hit":28672,"new_prefill":305,"score_linear":-28672.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":96882,"ongoing_decode_tokens":96882,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":948,"cache_hit":6144,"new_prefill":22833,"score_linear":90738.0,"score_lmetric":45666},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1104,"cache_hit":0,"new_prefill":28977,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":37007,"ongoing_decode_tokens":37007,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":28977,"score_linear":37007.0,"score_lmetric":28977},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":152426,"ongoing_decode_tokens":55130,"pending_prefill_tokens":2064,"num_requests":2,"active_p_offloads":0,"cached_blocks":1616,"cache_hit":1024,"new_prefill":27953,"score_linear":151402.0,"score_lmetric":60034},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":11725,"ongoing_decode_tokens":11725,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":835,"cache_hit":0,"new_prefill":28977,"score_linear":11725.0,"score_lmetric":28977},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":932,"cache_hit":0,"new_prefill":28977,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":191009,"ongoing_decode_tokens":191009,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":842,"cache_hit":0,"new_prefill":28977,"score_linear":191009.0,"score_lmetric":57954}],"chosen_score_linear":-28672.0,"chosen_score_lmetric":0,"t_first_token":348632.234128064,"t_first_token_unix":1779779497.248071,"t_done":348635.306401008,"t_done_unix":1779779500.3203447},{"request_id":"1329110:2:1331399:402","session_id":"1329110","input_length":12781,"t_proxy_recv":348635.380703886,"t_decision_unix":1779779500.3946455,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":4,"chosen_idx":2,"affinity_cache_hit":9216,"affinity_cache_ratio":0.7210703387841327,"affinity_num_requests":3,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":12781,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":828,"cache_hit":0,"new_prefill":12781,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":0,"new_prefill":12781,"score_linear":62618.0,"score_lmetric":12781},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1111,"cache_hit":0,"new_prefill":12781,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":12781,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":178114,"ongoing_decode_tokens":55130,"pending_prefill_tokens":12904,"num_requests":3,"active_p_offloads":0,"cached_blocks":1616,"cache_hit":9216,"new_prefill":3565,"score_linear":168898.0,"score_lmetric":49407},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":858,"cache_hit":0,"new_prefill":12781,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":932,"cache_hit":0,"new_prefill":12781,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":846,"cache_hit":0,"new_prefill":12781,"score_linear":121968.0,"score_lmetric":12781}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348636.437909557,"t_first_token_unix":1779779501.4518533,"t_done":348636.811088788,"t_done_unix":1779779501.8250325},{"request_id":"1330130:2:1331784:403","session_id":"1330130","input_length":14067,"t_proxy_recv":348636.575407484,"t_decision_unix":1779779501.5893483,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":11776,"affinity_cache_ratio":0.8371365607450061,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":11776,"estimated_new_tokens":2291,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":828,"cache_hit":0,"new_prefill":14067,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":0,"new_prefill":14067,"score_linear":62618.0,"score_lmetric":14067},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":12781,"ongoing_decode_tokens":12781,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1111,"cache_hit":0,"new_prefill":14067,"score_linear":12781.0,"score_lmetric":14067},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":14067,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":178114,"ongoing_decode_tokens":55130,"pending_prefill_tokens":12904,"num_requests":3,"active_p_offloads":0,"cached_blocks":1616,"cache_hit":0,"new_prefill":14067,"score_linear":178114.0,"score_lmetric":80913},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":858,"cache_hit":11776,"new_prefill":2291,"score_linear":-11776.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":932,"cache_hit":0,"new_prefill":14067,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":846,"cache_hit":0,"new_prefill":14067,"score_linear":121968.0,"score_lmetric":14067}],"chosen_score_linear":-11776.0,"chosen_score_lmetric":0,"t_first_token":348636.937125751,"t_first_token_unix":1779779501.951069,"t_done":348637.077283514,"t_done_unix":1779779502.0912266},{"request_id":"1329470:2:1332133:409","session_id":"1329470","input_length":11766,"t_proxy_recv":348637.739224987,"t_decision_unix":1779779502.7531667,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":10752,"affinity_cache_ratio":0.9138194798572157,"affinity_num_requests":0,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":10752,"estimated_new_tokens":1014,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":12421,"ongoing_decode_tokens":0,"pending_prefill_tokens":12421,"num_requests":1,"active_p_offloads":0,"cached_blocks":828,"cache_hit":0,"new_prefill":11766,"score_linear":12421.0,"score_lmetric":24187},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":0,"new_prefill":11766,"score_linear":62618.0,"score_lmetric":11766},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":73234,"ongoing_decode_tokens":0,"pending_prefill_tokens":6674,"num_requests":1,"active_p_offloads":0,"cached_blocks":1136,"cache_hit":0,"new_prefill":11766,"score_linear":73234.0,"score_lmetric":18440},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":11766,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":178114,"ongoing_decode_tokens":55130,"pending_prefill_tokens":12904,"num_requests":3,"active_p_offloads":0,"cached_blocks":1616,"cache_hit":0,"new_prefill":11766,"score_linear":178114.0,"score_lmetric":74010},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":103822,"ongoing_decode_tokens":0,"pending_prefill_tokens":21902,"num_requests":2,"active_p_offloads":0,"cached_blocks":862,"cache_hit":0,"new_prefill":11766,"score_linear":103822.0,"score_lmetric":67336},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":932,"cache_hit":10752,"new_prefill":1014,"score_linear":-10752.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":193745,"ongoing_decode_tokens":121968,"pending_prefill_tokens":2657,"num_requests":2,"active_p_offloads":0,"cached_blocks":846,"cache_hit":0,"new_prefill":11766,"score_linear":193745.0,"score_lmetric":28846}],"chosen_score_linear":-10752.0,"chosen_score_lmetric":0,"t_first_token":348637.872819568,"t_first_token_unix":1779779502.8867629,"t_done":348637.937988725,"t_done_unix":1779779502.951932},{"request_id":"1332021:1:1332021:407","session_id":"1332021","input_length":12421,"t_proxy_recv":348637.375235507,"t_decision_unix":1779779502.389177,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":12421,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":828,"cache_hit":0,"new_prefill":12421,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":0,"new_prefill":12421,"score_linear":62618.0,"score_lmetric":12421},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":73234,"ongoing_decode_tokens":0,"pending_prefill_tokens":6674,"num_requests":1,"active_p_offloads":0,"cached_blocks":1136,"cache_hit":0,"new_prefill":12421,"score_linear":73234.0,"score_lmetric":19095},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":12421,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":178114,"ongoing_decode_tokens":55130,"pending_prefill_tokens":12904,"num_requests":3,"active_p_offloads":0,"cached_blocks":1616,"cache_hit":0,"new_prefill":12421,"score_linear":178114.0,"score_lmetric":75975},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":21680,"ongoing_decode_tokens":0,"pending_prefill_tokens":21680,"num_requests":1,"active_p_offloads":0,"cached_blocks":862,"cache_hit":0,"new_prefill":12421,"score_linear":21680.0,"score_lmetric":34101},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":932,"cache_hit":0,"new_prefill":12421,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":193745,"ongoing_decode_tokens":121968,"pending_prefill_tokens":2657,"num_requests":2,"active_p_offloads":0,"cached_blocks":846,"cache_hit":0,"new_prefill":12421,"score_linear":193745.0,"score_lmetric":30156}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348638.395341262,"t_first_token_unix":1779779503.409285,"t_done":348638.654256142,"t_done_unix":1779779503.6681995},{"request_id":"1332472:1:1332472:410","session_id":"1332472","input_length":1810,"t_proxy_recv":348639.031299847,"t_decision_unix":1779779504.0452406,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":1810,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":852,"cache_hit":0,"new_prefill":1810,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":0,"new_prefill":1810,"score_linear":62618.0,"score_lmetric":1810},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":73234,"ongoing_decode_tokens":0,"pending_prefill_tokens":6674,"num_requests":1,"active_p_offloads":0,"cached_blocks":1136,"cache_hit":0,"new_prefill":1810,"score_linear":73234.0,"score_lmetric":8484},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":1810,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":178114,"ongoing_decode_tokens":55130,"pending_prefill_tokens":12904,"num_requests":3,"active_p_offloads":0,"cached_blocks":1616,"cache_hit":0,"new_prefill":1810,"score_linear":178114.0,"score_lmetric":44142},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":103822,"ongoing_decode_tokens":0,"pending_prefill_tokens":21902,"num_requests":2,"active_p_offloads":0,"cached_blocks":862,"cache_hit":0,"new_prefill":1810,"score_linear":103822.0,"score_lmetric":47424},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":934,"cache_hit":0,"new_prefill":1810,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":193745,"ongoing_decode_tokens":193745,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":846,"cache_hit":0,"new_prefill":1810,"score_linear":193745.0,"score_lmetric":3620}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348639.152550646,"t_first_token_unix":1779779504.1664944,"t_done":348639.854423417,"t_done_unix":1779779504.8683672},{"request_id":"1325541:2:1331846:404","session_id":"1325541","input_length":73234,"t_proxy_recv":348636.784072052,"t_decision_unix":1779779501.7980132,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":66560,"affinity_cache_ratio":0.9088674659311249,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":66560,"estimated_new_tokens":6674,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":828,"cache_hit":0,"new_prefill":73234,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":0,"new_prefill":73234,"score_linear":62618.0,"score_lmetric":73234},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":12781,"ongoing_decode_tokens":12781,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1111,"cache_hit":66560,"new_prefill":6674,"score_linear":-53779.0,"score_lmetric":6674},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":73234,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":178114,"ongoing_decode_tokens":55130,"pending_prefill_tokens":12904,"num_requests":3,"active_p_offloads":0,"cached_blocks":1616,"cache_hit":0,"new_prefill":73234,"score_linear":178114.0,"score_lmetric":258414},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":14067,"ongoing_decode_tokens":0,"pending_prefill_tokens":2291,"num_requests":1,"active_p_offloads":0,"cached_blocks":858,"cache_hit":0,"new_prefill":73234,"score_linear":14067.0,"score_lmetric":75525},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":932,"cache_hit":0,"new_prefill":73234,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":846,"cache_hit":0,"new_prefill":73234,"score_linear":121968.0,"score_lmetric":73234}],"chosen_score_linear":-53779.0,"chosen_score_lmetric":6674,"t_first_token":348639.701500971,"t_first_token_unix":1779779504.7154448,"t_done":348639.951189225,"t_done_unix":1779779504.965133},{"request_id":"1329110:3:1332476:411","session_id":"1329110","input_length":13479,"t_proxy_recv":348639.056129789,"t_decision_unix":1779779504.0700712,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":12800,"affinity_cache_ratio":0.9496253431263447,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":12800,"estimated_new_tokens":679,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":852,"cache_hit":0,"new_prefill":13479,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":0,"new_prefill":13479,"score_linear":62618.0,"score_lmetric":13479},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":73234,"ongoing_decode_tokens":0,"pending_prefill_tokens":6674,"num_requests":1,"active_p_offloads":0,"cached_blocks":1136,"cache_hit":12800,"new_prefill":679,"score_linear":60434.0,"score_lmetric":7353},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":1810,"ongoing_decode_tokens":0,"pending_prefill_tokens":1810,"num_requests":1,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":13479,"score_linear":1810.0,"score_lmetric":15289},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":178114,"ongoing_decode_tokens":55130,"pending_prefill_tokens":12904,"num_requests":3,"active_p_offloads":0,"cached_blocks":1616,"cache_hit":9216,"new_prefill":4263,"score_linear":168898.0,"score_lmetric":51501},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":103822,"ongoing_decode_tokens":0,"pending_prefill_tokens":21902,"num_requests":2,"active_p_offloads":0,"cached_blocks":862,"cache_hit":0,"new_prefill":13479,"score_linear":103822.0,"score_lmetric":70762},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":934,"cache_hit":0,"new_prefill":13479,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":193745,"ongoing_decode_tokens":193745,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":846,"cache_hit":0,"new_prefill":13479,"score_linear":193745.0,"score_lmetric":26958}],"chosen_score_linear":60434.0,"chosen_score_lmetric":7353,"t_first_token":348639.847676998,"t_first_token_unix":1779779504.861621,"t_done":348640.259255131,"t_done_unix":1779779505.2731986},{"request_id":"1332013:1:1332013:406","session_id":"1332013","input_length":21680,"t_proxy_recv":348637.358443613,"t_decision_unix":1779779502.3723848,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":21680,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":828,"cache_hit":0,"new_prefill":21680,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":0,"new_prefill":21680,"score_linear":62618.0,"score_lmetric":21680},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":73234,"ongoing_decode_tokens":0,"pending_prefill_tokens":6674,"num_requests":1,"active_p_offloads":0,"cached_blocks":1136,"cache_hit":0,"new_prefill":21680,"score_linear":73234.0,"score_lmetric":28354},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":21680,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":178114,"ongoing_decode_tokens":55130,"pending_prefill_tokens":12904,"num_requests":3,"active_p_offloads":0,"cached_blocks":1616,"cache_hit":12800,"new_prefill":8880,"score_linear":165314.0,"score_lmetric":65352},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":862,"cache_hit":0,"new_prefill":21680,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":932,"cache_hit":0,"new_prefill":21680,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":193745,"ongoing_decode_tokens":121968,"pending_prefill_tokens":2657,"num_requests":2,"active_p_offloads":0,"cached_blocks":846,"cache_hit":12800,"new_prefill":8880,"score_linear":180945.0,"score_lmetric":23074}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348639.8519856,"t_first_token_unix":1779779504.865929,"t_done":348641.977968758,"t_done_unix":1779779506.991912},{"request_id":"1332754:1:1332754:412","session_id":"1332754","input_length":19766,"t_proxy_recv":348640.059087624,"t_decision_unix":1779779505.0730288,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":19766,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":852,"cache_hit":0,"new_prefill":19766,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":0,"new_prefill":19766,"score_linear":62618.0,"score_lmetric":19766},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":13479,"ongoing_decode_tokens":13479,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1149,"cache_hit":0,"new_prefill":19766,"score_linear":13479.0,"score_lmetric":19766},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1007,"cache_hit":0,"new_prefill":19766,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":178114,"ongoing_decode_tokens":55130,"pending_prefill_tokens":12904,"num_requests":3,"active_p_offloads":0,"cached_blocks":1616,"cache_hit":0,"new_prefill":19766,"score_linear":178114.0,"score_lmetric":98010},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":103822,"ongoing_decode_tokens":103822,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":862,"cache_hit":0,"new_prefill":19766,"score_linear":103822.0,"score_lmetric":39532},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":934,"cache_hit":0,"new_prefill":19766,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":193745,"ongoing_decode_tokens":193745,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":846,"cache_hit":0,"new_prefill":19766,"score_linear":193745.0,"score_lmetric":39532}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348642.089381819,"t_first_token_unix":1779779507.1033251,"t_done":348642.571331563,"t_done_unix":1779779507.585275},{"request_id":"1269373:16:1331896:405","session_id":"1269373","input_length":71777,"t_proxy_recv":348636.975419885,"t_decision_unix":1779779501.989361,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":69120,"affinity_cache_ratio":0.9629825710185714,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":69120,"estimated_new_tokens":2657,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":828,"cache_hit":7680,"new_prefill":64097,"score_linear":-7680.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":0,"new_prefill":71777,"score_linear":62618.0,"score_lmetric":71777},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":73234,"ongoing_decode_tokens":0,"pending_prefill_tokens":6674,"num_requests":1,"active_p_offloads":0,"cached_blocks":1136,"cache_hit":0,"new_prefill":71777,"score_linear":73234.0,"score_lmetric":78451},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":71777,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":178114,"ongoing_decode_tokens":55130,"pending_prefill_tokens":12904,"num_requests":3,"active_p_offloads":0,"cached_blocks":1616,"cache_hit":7680,"new_prefill":64097,"score_linear":170434.0,"score_lmetric":231003},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":14067,"ongoing_decode_tokens":14067,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":858,"cache_hit":0,"new_prefill":71777,"score_linear":14067.0,"score_lmetric":71777},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":932,"cache_hit":7680,"new_prefill":64097,"score_linear":-7680.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":846,"cache_hit":69120,"new_prefill":2657,"score_linear":52848.0,"score_lmetric":2657}],"chosen_score_linear":52848.0,"chosen_score_lmetric":2657,"t_first_token":348638.140709448,"t_first_token_unix":1779779503.1546528,"t_done":348642.900861638,"t_done_unix":1779779507.914805},{"request_id":"1279412:18:1332945:413","session_id":"1279412","input_length":82488,"t_proxy_recv":348640.895427681,"t_decision_unix":1779779505.9093692,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":81408,"affinity_cache_ratio":0.9869071864998545,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":81408,"estimated_new_tokens":1080,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":852,"cache_hit":81408,"new_prefill":1080,"score_linear":-81408.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":0,"new_prefill":82488,"score_linear":62618.0,"score_lmetric":82488},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1150,"cache_hit":0,"new_prefill":82488,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1007,"cache_hit":0,"new_prefill":82488,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":178114,"ongoing_decode_tokens":55130,"pending_prefill_tokens":12904,"num_requests":3,"active_p_offloads":0,"cached_blocks":1616,"cache_hit":0,"new_prefill":82488,"score_linear":178114.0,"score_lmetric":286176},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":103822,"ongoing_decode_tokens":103822,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":862,"cache_hit":0,"new_prefill":82488,"score_linear":103822.0,"score_lmetric":164976},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":19766,"ongoing_decode_tokens":0,"pending_prefill_tokens":19766,"num_requests":1,"active_p_offloads":0,"cached_blocks":934,"cache_hit":12800,"new_prefill":69688,"score_linear":6966.0,"score_lmetric":89454},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":193745,"ongoing_decode_tokens":193745,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":846,"cache_hit":12800,"new_prefill":69688,"score_linear":180945.0,"score_lmetric":139376}],"chosen_score_linear":-81408.0,"chosen_score_lmetric":0,"t_first_token":348641.317383263,"t_first_token_unix":1779779506.3313289,"t_done":348644.093725967,"t_done_unix":1779779509.1076698},{"request_id":"1316848:11:1333994:416","session_id":"1316848","input_length":29656,"t_proxy_recv":348644.433771843,"t_decision_unix":1779779509.4477131,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":29184,"affinity_cache_ratio":0.9840841650930672,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":29184,"estimated_new_tokens":472,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":854,"cache_hit":29184,"new_prefill":472,"score_linear":-29184.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":6144,"new_prefill":23512,"score_linear":56474.0,"score_lmetric":23512},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1150,"cache_hit":0,"new_prefill":29656,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1007,"cache_hit":0,"new_prefill":29656,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":178114,"ongoing_decode_tokens":55130,"pending_prefill_tokens":12904,"num_requests":3,"active_p_offloads":0,"cached_blocks":1616,"cache_hit":1024,"new_prefill":28632,"score_linear":177090.0,"score_lmetric":124608},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":82142,"ongoing_decode_tokens":82142,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":904,"cache_hit":0,"new_prefill":29656,"score_linear":82142.0,"score_lmetric":29656},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":972,"cache_hit":0,"new_prefill":29656,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":851,"cache_hit":0,"new_prefill":29656,"score_linear":121968.0,"score_lmetric":29656}],"chosen_score_linear":-29184.0,"chosen_score_lmetric":0,"t_first_token":348644.552272921,"t_first_token_unix":1779779509.5662165,"t_done":348644.842290513,"t_done_unix":1779779509.8562338},{"request_id":"1317598:4:1332127:408","session_id":"1317598","input_length":82142,"t_proxy_recv":348637.730087873,"t_decision_unix":1779779502.7440271,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":81920,"affinity_cache_ratio":0.9972973631029194,"affinity_num_requests":1,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":81920,"estimated_new_tokens":222,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":12421,"ongoing_decode_tokens":0,"pending_prefill_tokens":12421,"num_requests":1,"active_p_offloads":0,"cached_blocks":828,"cache_hit":0,"new_prefill":82142,"score_linear":12421.0,"score_lmetric":94563},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":0,"new_prefill":82142,"score_linear":62618.0,"score_lmetric":82142},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":73234,"ongoing_decode_tokens":0,"pending_prefill_tokens":6674,"num_requests":1,"active_p_offloads":0,"cached_blocks":1136,"cache_hit":0,"new_prefill":82142,"score_linear":73234.0,"score_lmetric":88816},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":82142,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":178114,"ongoing_decode_tokens":55130,"pending_prefill_tokens":12904,"num_requests":3,"active_p_offloads":0,"cached_blocks":1616,"cache_hit":512,"new_prefill":81630,"score_linear":177602.0,"score_lmetric":283602},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":21680,"ongoing_decode_tokens":0,"pending_prefill_tokens":21680,"num_requests":1,"active_p_offloads":0,"cached_blocks":862,"cache_hit":81920,"new_prefill":222,"score_linear":-60240.0,"score_lmetric":21902},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":932,"cache_hit":0,"new_prefill":82142,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":193745,"ongoing_decode_tokens":121968,"pending_prefill_tokens":2657,"num_requests":2,"active_p_offloads":0,"cached_blocks":846,"cache_hit":0,"new_prefill":82142,"score_linear":193745.0,"score_lmetric":169598}],"chosen_score_linear":-60240.0,"chosen_score_lmetric":21902,"t_first_token":348639.854849759,"t_first_token_unix":1779779504.8687928,"t_done":348644.864428556,"t_done_unix":1779779509.878372},{"request_id":"1334223:1:1334223:417","session_id":"1334223","input_length":23689,"t_proxy_recv":348645.305970848,"t_decision_unix":1779779510.319912,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":23689,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":855,"cache_hit":0,"new_prefill":23689,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":0,"new_prefill":23689,"score_linear":62618.0,"score_lmetric":23689},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1150,"cache_hit":0,"new_prefill":23689,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1007,"cache_hit":0,"new_prefill":23689,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":178114,"ongoing_decode_tokens":55130,"pending_prefill_tokens":12904,"num_requests":3,"active_p_offloads":0,"cached_blocks":1616,"cache_hit":0,"new_prefill":23689,"score_linear":178114.0,"score_lmetric":109779},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":905,"cache_hit":0,"new_prefill":23689,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":972,"cache_hit":0,"new_prefill":23689,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":851,"cache_hit":0,"new_prefill":23689,"score_linear":121968.0,"score_lmetric":23689}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348647.970618626,"t_first_token_unix":1779779512.9845622,"t_done":348649.133957935,"t_done_unix":1779779514.1479023},{"request_id":"1309252:2:1334746:419","session_id":"1309252","input_length":45601,"t_proxy_recv":348647.275846314,"t_decision_unix":1779779512.2897873,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":44544,"affinity_cache_ratio":0.9768206837569352,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":44544,"estimated_new_tokens":1057,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":855,"cache_hit":0,"new_prefill":45601,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":0,"new_prefill":45601,"score_linear":62618.0,"score_lmetric":45601},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":23689,"ongoing_decode_tokens":0,"pending_prefill_tokens":23689,"num_requests":1,"active_p_offloads":0,"cached_blocks":1150,"cache_hit":44544,"new_prefill":1057,"score_linear":-20855.0,"score_lmetric":24746},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1007,"cache_hit":0,"new_prefill":45601,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":178114,"ongoing_decode_tokens":55130,"pending_prefill_tokens":12904,"num_requests":3,"active_p_offloads":0,"cached_blocks":1616,"cache_hit":0,"new_prefill":45601,"score_linear":178114.0,"score_lmetric":175515},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":905,"cache_hit":0,"new_prefill":45601,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":972,"cache_hit":0,"new_prefill":45601,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":851,"cache_hit":0,"new_prefill":45601,"score_linear":121968.0,"score_lmetric":45601}],"chosen_score_linear":-20855.0,"chosen_score_lmetric":24746,"t_first_token":348648.277964983,"t_first_token_unix":1779779513.2919083,"t_done":348650.056978319,"t_done_unix":1779779515.0709226},{"request_id":"1330881:2:1336222:422","session_id":"1330881","input_length":11989,"t_proxy_recv":348652.388638782,"t_decision_unix":1779779517.4025803,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":3584,"affinity_cache_ratio":0.29894069563766784,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":3584,"estimated_new_tokens":8405,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":131928,"ongoing_decode_tokens":44664,"pending_prefill_tokens":4832,"num_requests":2,"active_p_offloads":0,"cached_blocks":855,"cache_hit":0,"new_prefill":11989,"score_linear":131928.0,"score_lmetric":33642},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":0,"new_prefill":11989,"score_linear":62618.0,"score_lmetric":11989},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1198,"cache_hit":3584,"new_prefill":8405,"score_linear":-3584.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1007,"cache_hit":0,"new_prefill":11989,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":178114,"ongoing_decode_tokens":55130,"pending_prefill_tokens":12904,"num_requests":3,"active_p_offloads":0,"cached_blocks":1616,"cache_hit":0,"new_prefill":11989,"score_linear":178114.0,"score_lmetric":74679},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":61229,"ongoing_decode_tokens":0,"pending_prefill_tokens":57645,"num_requests":1,"active_p_offloads":0,"cached_blocks":905,"cache_hit":0,"new_prefill":11989,"score_linear":61229.0,"score_lmetric":69634},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":972,"cache_hit":0,"new_prefill":11989,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":851,"cache_hit":0,"new_prefill":11989,"score_linear":121968.0,"score_lmetric":11989}],"chosen_score_linear":-3584.0,"chosen_score_lmetric":0,"t_first_token":348653.129724984,"t_first_token_unix":1779779518.1436682,"t_done":348653.291674818,"t_done_unix":1779779518.3056185},{"request_id":"1266668:12:1303535:276","session_id":"1266668","input_length":121968,"t_proxy_recv":348561.095426639,"t_decision_unix":1779779426.109368,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":97792,"affinity_cache_ratio":0.8017840745113473,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":97792,"estimated_new_tokens":24176,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":64558,"ongoing_decode_tokens":64558,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":660,"cache_hit":7680,"new_prefill":114288,"score_linear":56878.0,"score_lmetric":114288},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":75826,"ongoing_decode_tokens":0,"pending_prefill_tokens":75826,"num_requests":1,"active_p_offloads":0,"cached_blocks":778,"cache_hit":0,"new_prefill":121968,"score_linear":75826.0,"score_lmetric":197794},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":24545,"ongoing_decode_tokens":24545,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":811,"cache_hit":0,"new_prefill":121968,"score_linear":24545.0,"score_lmetric":121968},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":896,"cache_hit":0,"new_prefill":121968,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1132,"cache_hit":0,"new_prefill":121968,"score_linear":0.0,"score_lmetric":-121968},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":43607,"ongoing_decode_tokens":43607,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":548,"cache_hit":0,"new_prefill":121968,"score_linear":43607.0,"score_lmetric":121968},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":801,"cache_hit":7680,"new_prefill":114288,"score_linear":-7680.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":50680,"ongoing_decode_tokens":50680,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":795,"cache_hit":97792,"new_prefill":24176,"score_linear":-47112.0,"score_lmetric":24176}],"chosen_score_linear":-47112.0,"chosen_score_lmetric":24176,"t_first_token":348577.015460469,"t_first_token_unix":1779779442.0294044,"t_done":348653.576789184,"t_done_unix":1779779518.5907333},{"request_id":"1316848:12:1334743:418","session_id":"1316848","input_length":44664,"t_proxy_recv":348647.283344536,"t_decision_unix":1779779512.297286,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":29696,"affinity_cache_ratio":0.6648755149561167,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":29696,"estimated_new_tokens":14968,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":855,"cache_hit":29696,"new_prefill":14968,"score_linear":-29696.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":6144,"new_prefill":38520,"score_linear":56474.0,"score_lmetric":38520},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":69290,"ongoing_decode_tokens":0,"pending_prefill_tokens":24746,"num_requests":2,"active_p_offloads":0,"cached_blocks":1150,"cache_hit":0,"new_prefill":44664,"score_linear":69290.0,"score_lmetric":138820},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1007,"cache_hit":0,"new_prefill":44664,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":178114,"ongoing_decode_tokens":55130,"pending_prefill_tokens":12904,"num_requests":3,"active_p_offloads":0,"cached_blocks":1616,"cache_hit":1024,"new_prefill":43640,"score_linear":177090.0,"score_lmetric":169632},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":905,"cache_hit":0,"new_prefill":44664,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":972,"cache_hit":0,"new_prefill":44664,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":851,"cache_hit":0,"new_prefill":44664,"score_linear":121968.0,"score_lmetric":44664}],"chosen_score_linear":-29696.0,"chosen_score_lmetric":0,"t_first_token":348651.129361996,"t_first_token_unix":1779779516.1433055,"t_done":348655.19797484,"t_done_unix":1779779520.211918},{"request_id":"1336620:1:1336620:423","session_id":"1336620","input_length":14666,"t_proxy_recv":348653.691103518,"t_decision_unix":1779779518.705045,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":7680,"estimated_new_tokens":6986,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":131928,"ongoing_decode_tokens":131928,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":855,"cache_hit":7680,"new_prefill":6986,"score_linear":124248.0,"score_lmetric":13972},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":0,"new_prefill":14666,"score_linear":62618.0,"score_lmetric":14666},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1214,"cache_hit":0,"new_prefill":14666,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1007,"cache_hit":0,"new_prefill":14666,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":178114,"ongoing_decode_tokens":55130,"pending_prefill_tokens":12904,"num_requests":3,"active_p_offloads":0,"cached_blocks":1616,"cache_hit":7680,"new_prefill":6986,"score_linear":170434.0,"score_lmetric":59670},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":61229,"ongoing_decode_tokens":0,"pending_prefill_tokens":57645,"num_requests":1,"active_p_offloads":0,"cached_blocks":905,"cache_hit":0,"new_prefill":14666,"score_linear":61229.0,"score_lmetric":72311},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":972,"cache_hit":7680,"new_prefill":6986,"score_linear":-7680.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":900,"cache_hit":7680,"new_prefill":6986,"score_linear":-7680.0,"score_lmetric":0}],"chosen_score_linear":-7680.0,"chosen_score_lmetric":0,"t_first_token":348654.469031594,"t_first_token_unix":1779779519.482975,"t_done":348655.438620147,"t_done_unix":1779779520.4525635},{"request_id":"1279412:19:1335444:421","session_id":"1279412","input_length":87264,"t_proxy_recv":348649.576870377,"t_decision_unix":1779779514.5908115,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":82432,"affinity_cache_ratio":0.9446277961129447,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":82432,"estimated_new_tokens":4832,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":44664,"ongoing_decode_tokens":0,"pending_prefill_tokens":14968,"num_requests":1,"active_p_offloads":0,"cached_blocks":855,"cache_hit":82432,"new_prefill":4832,"score_linear":-37768.0,"score_lmetric":19800},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":0,"new_prefill":87264,"score_linear":62618.0,"score_lmetric":87264},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":45601,"ongoing_decode_tokens":45601,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1196,"cache_hit":0,"new_prefill":87264,"score_linear":45601.0,"score_lmetric":87264},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1007,"cache_hit":0,"new_prefill":87264,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":178114,"ongoing_decode_tokens":55130,"pending_prefill_tokens":12904,"num_requests":3,"active_p_offloads":0,"cached_blocks":1616,"cache_hit":0,"new_prefill":87264,"score_linear":178114.0,"score_lmetric":300504},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":61229,"ongoing_decode_tokens":0,"pending_prefill_tokens":57645,"num_requests":1,"active_p_offloads":0,"cached_blocks":905,"cache_hit":0,"new_prefill":87264,"score_linear":61229.0,"score_lmetric":144909},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":972,"cache_hit":12800,"new_prefill":74464,"score_linear":-12800.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":851,"cache_hit":12800,"new_prefill":74464,"score_linear":109168.0,"score_lmetric":74464}],"chosen_score_linear":-37768.0,"chosen_score_lmetric":19800,"t_first_token":348653.560388975,"t_first_token_unix":1779779518.574333,"t_done":348655.99342504,"t_done_unix":1779779521.0073693},{"request_id":"1337035:1:1337035:425","session_id":"1337035","input_length":12644,"t_proxy_recv":348655.189541498,"t_decision_unix":1779779520.2034826,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":12644,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":131928,"ongoing_decode_tokens":131928,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":855,"cache_hit":0,"new_prefill":12644,"score_linear":131928.0,"score_lmetric":25288},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":0,"new_prefill":12644,"score_linear":62618.0,"score_lmetric":12644},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1214,"cache_hit":0,"new_prefill":12644,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1007,"cache_hit":0,"new_prefill":12644,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":178114,"ongoing_decode_tokens":55130,"pending_prefill_tokens":12904,"num_requests":3,"active_p_offloads":0,"cached_blocks":1616,"cache_hit":0,"new_prefill":12644,"score_linear":178114.0,"score_lmetric":76644},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":61229,"ongoing_decode_tokens":0,"pending_prefill_tokens":57645,"num_requests":1,"active_p_offloads":0,"cached_blocks":905,"cache_hit":0,"new_prefill":12644,"score_linear":61229.0,"score_lmetric":70289},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":20009,"ongoing_decode_tokens":0,"pending_prefill_tokens":5673,"num_requests":1,"active_p_offloads":0,"cached_blocks":972,"cache_hit":0,"new_prefill":12644,"score_linear":20009.0,"score_lmetric":18317},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":14666,"ongoing_decode_tokens":14666,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":900,"cache_hit":0,"new_prefill":12644,"score_linear":14666.0,"score_lmetric":12644}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348656.236296423,"t_first_token_unix":1779779521.25024,"t_done":348656.415152391,"t_done_unix":1779779521.4290957},{"request_id":"1259178:2:1336768:424","session_id":"1259178","input_length":20009,"t_proxy_recv":348654.157194964,"t_decision_unix":1779779519.171136,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":14336,"affinity_cache_ratio":0.716477585086711,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":14336,"estimated_new_tokens":5673,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":131928,"ongoing_decode_tokens":131928,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":855,"cache_hit":0,"new_prefill":20009,"score_linear":131928.0,"score_lmetric":40018},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":0,"new_prefill":20009,"score_linear":62618.0,"score_lmetric":20009},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1214,"cache_hit":0,"new_prefill":20009,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1007,"cache_hit":0,"new_prefill":20009,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":178114,"ongoing_decode_tokens":55130,"pending_prefill_tokens":12904,"num_requests":3,"active_p_offloads":0,"cached_blocks":1616,"cache_hit":0,"new_prefill":20009,"score_linear":178114.0,"score_lmetric":98739},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":61229,"ongoing_decode_tokens":0,"pending_prefill_tokens":57645,"num_requests":1,"active_p_offloads":0,"cached_blocks":905,"cache_hit":0,"new_prefill":20009,"score_linear":61229.0,"score_lmetric":77654},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":972,"cache_hit":14336,"new_prefill":5673,"score_linear":-14336.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":14666,"ongoing_decode_tokens":0,"pending_prefill_tokens":6986,"num_requests":1,"active_p_offloads":0,"cached_blocks":900,"cache_hit":0,"new_prefill":20009,"score_linear":14666.0,"score_lmetric":26995}],"chosen_score_linear":-14336.0,"chosen_score_lmetric":0,"t_first_token":348656.229319154,"t_first_token_unix":1779779521.2432635,"t_done":348657.039526163,"t_done_unix":1779779522.0534697},{"request_id":"1259178:3:1337653:429","session_id":"1259178","input_length":22398,"t_proxy_recv":348657.473343071,"t_decision_unix":1779779522.4872847,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":19968,"affinity_cache_ratio":0.8915081703723546,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":19968,"estimated_new_tokens":2430,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":176906,"ongoing_decode_tokens":0,"pending_prefill_tokens":45322,"num_requests":3,"active_p_offloads":0,"cached_blocks":893,"cache_hit":0,"new_prefill":22398,"score_linear":176906.0,"score_lmetric":203160},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":0,"new_prefill":22398,"score_linear":62618.0,"score_lmetric":22398},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1238,"cache_hit":0,"new_prefill":22398,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1007,"cache_hit":0,"new_prefill":22398,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":178114,"ongoing_decode_tokens":152426,"pending_prefill_tokens":10840,"num_requests":3,"active_p_offloads":0,"cached_blocks":1616,"cache_hit":0,"new_prefill":22398,"score_linear":178114.0,"score_lmetric":99714},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":61229,"ongoing_decode_tokens":0,"pending_prefill_tokens":57645,"num_requests":1,"active_p_offloads":0,"cached_blocks":905,"cache_hit":0,"new_prefill":22398,"score_linear":61229.0,"score_lmetric":80043},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":983,"cache_hit":19968,"new_prefill":2430,"score_linear":-19968.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":913,"cache_hit":0,"new_prefill":22398,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-19968.0,"chosen_score_lmetric":0,"t_first_token":348657.92348634,"t_first_token_unix":1779779522.9374297,"t_done":348658.780760227,"t_done_unix":1779779523.7947042},{"request_id":"1257323:2:1329541:393","session_id":"1257323","input_length":90605,"t_proxy_recv":348628.946640653,"t_decision_unix":1779779493.9605818,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":90112,"affinity_cache_ratio":0.9945587991832681,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":90112,"estimated_new_tokens":493,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":81008,"ongoing_decode_tokens":81008,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":816,"cache_hit":0,"new_prefill":90605,"score_linear":81008.0,"score_lmetric":90605},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":96882,"ongoing_decode_tokens":96882,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":948,"cache_hit":0,"new_prefill":90605,"score_linear":96882.0,"score_lmetric":181210},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":79952,"ongoing_decode_tokens":0,"pending_prefill_tokens":68176,"num_requests":2,"active_p_offloads":0,"cached_blocks":972,"cache_hit":0,"new_prefill":90605,"score_linear":79952.0,"score_lmetric":317562},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":36880,"ongoing_decode_tokens":0,"pending_prefill_tokens":36880,"num_requests":1,"active_p_offloads":0,"cached_blocks":932,"cache_hit":0,"new_prefill":90605,"score_linear":36880.0,"score_lmetric":127485},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1616,"cache_hit":90112,"new_prefill":493,"score_linear":-34982.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":835,"cache_hit":512,"new_prefill":90093,"score_linear":-512.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":10847,"ongoing_decode_tokens":0,"pending_prefill_tokens":10847,"num_requests":1,"active_p_offloads":0,"cached_blocks":911,"cache_hit":0,"new_prefill":90605,"score_linear":10847.0,"score_lmetric":101452},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":191009,"ongoing_decode_tokens":191009,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":842,"cache_hit":0,"new_prefill":90605,"score_linear":191009.0,"score_lmetric":181210}],"chosen_score_linear":-34982.0,"chosen_score_lmetric":0,"t_first_token":348657.469022001,"t_first_token_unix":1779779522.482966,"t_done":348658.798417038,"t_done_unix":1779779523.8123608},{"request_id":"1324964:5:1330169:397","session_id":"1324964","input_length":6691,"t_proxy_recv":348630.872370953,"t_decision_unix":1779779495.8863122,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":5120,"affinity_cache_ratio":0.7652069944701838,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":5120,"estimated_new_tokens":1571,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":827,"cache_hit":0,"new_prefill":6691,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":96882,"ongoing_decode_tokens":96882,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":948,"cache_hit":0,"new_prefill":6691,"score_linear":96882.0,"score_lmetric":13382},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1104,"cache_hit":0,"new_prefill":6691,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":36880,"ongoing_decode_tokens":0,"pending_prefill_tokens":36880,"num_requests":1,"active_p_offloads":0,"cached_blocks":932,"cache_hit":0,"new_prefill":6691,"score_linear":36880.0,"score_lmetric":43571},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":145735,"ongoing_decode_tokens":55130,"pending_prefill_tokens":493,"num_requests":1,"active_p_offloads":0,"cached_blocks":1616,"cache_hit":5120,"new_prefill":1571,"score_linear":140615.0,"score_lmetric":2064},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":11725,"ongoing_decode_tokens":0,"pending_prefill_tokens":11725,"num_requests":1,"active_p_offloads":0,"cached_blocks":835,"cache_hit":0,"new_prefill":6691,"score_linear":11725.0,"score_lmetric":18416},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":10847,"ongoing_decode_tokens":10847,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":911,"cache_hit":0,"new_prefill":6691,"score_linear":10847.0,"score_lmetric":6691},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":191009,"ongoing_decode_tokens":191009,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":842,"cache_hit":0,"new_prefill":6691,"score_linear":191009.0,"score_lmetric":13382}],"chosen_score_linear":140615.0,"chosen_score_lmetric":2064,"t_first_token":348657.470361565,"t_first_token_unix":1779779522.4843044,"t_done":348658.805547711,"t_done_unix":1779779523.8194911},{"request_id":"1326192:3:1330643:399","session_id":"1326192","input_length":25688,"t_proxy_recv":348632.67133642,"t_decision_unix":1779779497.685278,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":14848,"affinity_cache_ratio":0.5780130800373715,"affinity_num_requests":2,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":14848,"estimated_new_tokens":10840,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":28977,"ongoing_decode_tokens":28977,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":827,"cache_hit":0,"new_prefill":25688,"score_linear":28977.0,"score_lmetric":25688},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":96882,"ongoing_decode_tokens":96882,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":948,"cache_hit":0,"new_prefill":25688,"score_linear":96882.0,"score_lmetric":51376},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1104,"cache_hit":0,"new_prefill":25688,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":25688,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":152426,"ongoing_decode_tokens":55130,"pending_prefill_tokens":2064,"num_requests":2,"active_p_offloads":0,"cached_blocks":1616,"cache_hit":14848,"new_prefill":10840,"score_linear":137578.0,"score_lmetric":25808},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":858,"cache_hit":0,"new_prefill":25688,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":932,"cache_hit":0,"new_prefill":25688,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":191009,"ongoing_decode_tokens":191009,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":842,"cache_hit":0,"new_prefill":25688,"score_linear":191009.0,"score_lmetric":51376}],"chosen_score_linear":137578.0,"chosen_score_lmetric":25808,"t_first_token":348658.155684457,"t_first_token_unix":1779779523.1696281,"t_done":348658.89135169,"t_done_unix":1779779523.9052947},{"request_id":"1326192:4:1331370:401","session_id":"1326192","input_length":27049,"t_proxy_recv":348658.907285103,"t_decision_unix":1779779523.9212265,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":25600,"affinity_cache_ratio":0.9464305519612555,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":25600,"estimated_new_tokens":1449,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":176906,"ongoing_decode_tokens":0,"pending_prefill_tokens":45322,"num_requests":3,"active_p_offloads":0,"cached_blocks":893,"cache_hit":0,"new_prefill":27049,"score_linear":176906.0,"score_lmetric":217113},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":0,"new_prefill":27049,"score_linear":62618.0,"score_lmetric":27049},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1238,"cache_hit":0,"new_prefill":27049,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1007,"cache_hit":0,"new_prefill":27049,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1641,"cache_hit":25600,"new_prefill":1449,"score_linear":29530.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":144317,"ongoing_decode_tokens":0,"pending_prefill_tokens":58301,"num_requests":2,"active_p_offloads":0,"cached_blocks":905,"cache_hit":0,"new_prefill":27049,"score_linear":144317.0,"score_lmetric":170700},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":988,"cache_hit":0,"new_prefill":27049,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":913,"cache_hit":0,"new_prefill":27049,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":29530.0,"chosen_score_lmetric":0,"t_first_token":348659.255842511,"t_first_token_unix":1779779524.2697861,"t_done":348659.833523099,"t_done_unix":1779779524.8474665},{"request_id":"1338292:1:1338292:432","session_id":"1338292","input_length":9325,"t_proxy_recv":348659.670678537,"t_decision_unix":1779779524.6846197,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":9325,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":176906,"ongoing_decode_tokens":0,"pending_prefill_tokens":45322,"num_requests":3,"active_p_offloads":0,"cached_blocks":893,"cache_hit":0,"new_prefill":9325,"score_linear":176906.0,"score_lmetric":163941},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":0,"new_prefill":9325,"score_linear":62618.0,"score_lmetric":9325},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1238,"cache_hit":0,"new_prefill":9325,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1007,"cache_hit":0,"new_prefill":9325,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":82179,"ongoing_decode_tokens":82179,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1641,"cache_hit":0,"new_prefill":9325,"score_linear":82179.0,"score_lmetric":9325},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":144317,"ongoing_decode_tokens":0,"pending_prefill_tokens":58301,"num_requests":2,"active_p_offloads":0,"cached_blocks":905,"cache_hit":0,"new_prefill":9325,"score_linear":144317.0,"score_lmetric":135252},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":988,"cache_hit":0,"new_prefill":9325,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":73073,"ongoing_decode_tokens":0,"pending_prefill_tokens":1393,"num_requests":1,"active_p_offloads":0,"cached_blocks":913,"cache_hit":0,"new_prefill":9325,"score_linear":73073.0,"score_lmetric":10718}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348660.374937092,"t_first_token_unix":1779779525.388881,"t_done":348660.675548618,"t_done_unix":1779779525.689492},{"request_id":"1336620:2:1338440:434","session_id":"1336620","input_length":14918,"t_proxy_recv":348660.081914824,"t_decision_unix":1779779525.0958564,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":14336,"affinity_cache_ratio":0.960986727443357,"affinity_num_requests":1,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":14336,"estimated_new_tokens":582,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":176906,"ongoing_decode_tokens":0,"pending_prefill_tokens":45322,"num_requests":3,"active_p_offloads":0,"cached_blocks":893,"cache_hit":7680,"new_prefill":7238,"score_linear":169226.0,"score_lmetric":157680},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":0,"new_prefill":14918,"score_linear":62618.0,"score_lmetric":14918},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1238,"cache_hit":0,"new_prefill":14918,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":59540,"ongoing_decode_tokens":0,"pending_prefill_tokens":9876,"num_requests":2,"active_p_offloads":0,"cached_blocks":1007,"cache_hit":0,"new_prefill":14918,"score_linear":59540.0,"score_lmetric":49588},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1643,"cache_hit":7680,"new_prefill":7238,"score_linear":47450.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":144317,"ongoing_decode_tokens":0,"pending_prefill_tokens":58301,"num_requests":2,"active_p_offloads":0,"cached_blocks":905,"cache_hit":0,"new_prefill":14918,"score_linear":144317.0,"score_lmetric":146438},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":988,"cache_hit":7680,"new_prefill":7238,"score_linear":-7680.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":73073,"ongoing_decode_tokens":73073,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":913,"cache_hit":14336,"new_prefill":582,"score_linear":58737.0,"score_lmetric":582}],"chosen_score_linear":58737.0,"chosen_score_lmetric":582,"t_first_token":348660.152637094,"t_first_token_unix":1779779525.1665804,"t_done":348661.822192547,"t_done_unix":1779779526.8361359},{"request_id":"1269373:17:1338104:431","session_id":"1269373","input_length":73073,"t_proxy_recv":348659.12956908,"t_decision_unix":1779779524.1435103,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":71680,"affinity_cache_ratio":0.9809368713478303,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":71680,"estimated_new_tokens":1393,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":176906,"ongoing_decode_tokens":0,"pending_prefill_tokens":45322,"num_requests":3,"active_p_offloads":0,"cached_blocks":893,"cache_hit":7680,"new_prefill":65393,"score_linear":169226.0,"score_lmetric":332145},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":0,"new_prefill":73073,"score_linear":62618.0,"score_lmetric":73073},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1238,"cache_hit":0,"new_prefill":73073,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1007,"cache_hit":0,"new_prefill":73073,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":82179,"ongoing_decode_tokens":55130,"pending_prefill_tokens":1449,"num_requests":1,"active_p_offloads":0,"cached_blocks":1641,"cache_hit":7680,"new_prefill":65393,"score_linear":74499.0,"score_lmetric":66842},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":144317,"ongoing_decode_tokens":0,"pending_prefill_tokens":58301,"num_requests":2,"active_p_offloads":0,"cached_blocks":905,"cache_hit":0,"new_prefill":73073,"score_linear":144317.0,"score_lmetric":262748},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":988,"cache_hit":7680,"new_prefill":65393,"score_linear":-7680.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":913,"cache_hit":71680,"new_prefill":1393,"score_linear":-71680.0,"score_lmetric":0}],"chosen_score_linear":-71680.0,"chosen_score_lmetric":0,"t_first_token":348659.791413303,"t_first_token_unix":1779779524.8053567,"t_done":348662.518976884,"t_done_unix":1779779527.5329204},{"request_id":"1317598:5:1337780:430","session_id":"1317598","input_length":83088,"t_proxy_recv":348657.946843595,"t_decision_unix":1779779522.9607847,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":82432,"affinity_cache_ratio":0.99210475640285,"affinity_num_requests":1,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":82432,"estimated_new_tokens":656,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":176906,"ongoing_decode_tokens":0,"pending_prefill_tokens":45322,"num_requests":3,"active_p_offloads":0,"cached_blocks":893,"cache_hit":0,"new_prefill":83088,"score_linear":176906.0,"score_lmetric":385230},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":0,"new_prefill":83088,"score_linear":62618.0,"score_lmetric":83088},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1238,"cache_hit":0,"new_prefill":83088,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1007,"cache_hit":0,"new_prefill":83088,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":178114,"ongoing_decode_tokens":152426,"pending_prefill_tokens":10840,"num_requests":3,"active_p_offloads":0,"cached_blocks":1616,"cache_hit":512,"new_prefill":82576,"score_linear":177602.0,"score_lmetric":280248},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":61229,"ongoing_decode_tokens":0,"pending_prefill_tokens":57645,"num_requests":1,"active_p_offloads":0,"cached_blocks":905,"cache_hit":82432,"new_prefill":656,"score_linear":-21203.0,"score_lmetric":58301},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":22398,"ongoing_decode_tokens":22398,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":983,"cache_hit":0,"new_prefill":83088,"score_linear":22398.0,"score_lmetric":83088},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":913,"cache_hit":0,"new_prefill":83088,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-21203.0,"chosen_score_lmetric":58301,"t_first_token":348662.130811328,"t_first_token_unix":1779779527.1447546,"t_done":348662.884480454,"t_done_unix":1779779527.8984241},{"request_id":"1253743:5:1338406:433","session_id":"1253743","input_length":50215,"t_proxy_recv":348659.998911628,"t_decision_unix":1779779525.0128527,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":49664,"affinity_cache_ratio":0.9890271831126157,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":49664,"estimated_new_tokens":551,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":176906,"ongoing_decode_tokens":0,"pending_prefill_tokens":45322,"num_requests":3,"active_p_offloads":0,"cached_blocks":893,"cache_hit":0,"new_prefill":50215,"score_linear":176906.0,"score_lmetric":286611},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":4608,"new_prefill":45607,"score_linear":58010.0,"score_lmetric":45607},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1238,"cache_hit":0,"new_prefill":50215,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":9325,"ongoing_decode_tokens":0,"pending_prefill_tokens":9325,"num_requests":1,"active_p_offloads":0,"cached_blocks":1007,"cache_hit":49664,"new_prefill":551,"score_linear":-40339.0,"score_lmetric":9876},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1643,"cache_hit":0,"new_prefill":50215,"score_linear":55130.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":144317,"ongoing_decode_tokens":0,"pending_prefill_tokens":58301,"num_requests":2,"active_p_offloads":0,"cached_blocks":905,"cache_hit":17408,"new_prefill":32807,"score_linear":126909.0,"score_lmetric":182216},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":988,"cache_hit":0,"new_prefill":50215,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":73073,"ongoing_decode_tokens":73073,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":913,"cache_hit":0,"new_prefill":50215,"score_linear":73073.0,"score_lmetric":50215}],"chosen_score_linear":-40339.0,"chosen_score_lmetric":9876,"t_first_token":348660.517336108,"t_first_token_unix":1779779525.5312796,"t_done":348663.576666425,"t_done_unix":1779779528.5906103},{"request_id":"1314357:2:1335184:420","session_id":"1314357","input_length":61229,"t_proxy_recv":348648.640250942,"t_decision_unix":1779779513.6541922,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":4,"chosen_idx":5,"affinity_cache_hit":56832,"affinity_cache_ratio":0.9281876235117347,"affinity_num_requests":3,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":3584,"estimated_new_tokens":57645,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":44664,"ongoing_decode_tokens":0,"pending_prefill_tokens":14968,"num_requests":1,"active_p_offloads":0,"cached_blocks":855,"cache_hit":0,"new_prefill":61229,"score_linear":44664.0,"score_lmetric":76197},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":0,"new_prefill":61229,"score_linear":62618.0,"score_lmetric":61229},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":69290,"ongoing_decode_tokens":69290,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1150,"cache_hit":0,"new_prefill":61229,"score_linear":69290.0,"score_lmetric":122458},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1007,"cache_hit":0,"new_prefill":61229,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":178114,"ongoing_decode_tokens":55130,"pending_prefill_tokens":12904,"num_requests":3,"active_p_offloads":0,"cached_blocks":1616,"cache_hit":56832,"new_prefill":4397,"score_linear":121282.0,"score_lmetric":51903},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":905,"cache_hit":3584,"new_prefill":57645,"score_linear":-3584.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":972,"cache_hit":0,"new_prefill":61229,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":121968,"ongoing_decode_tokens":121968,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":851,"cache_hit":0,"new_prefill":61229,"score_linear":121968.0,"score_lmetric":61229}],"chosen_score_linear":-3584.0,"chosen_score_lmetric":0,"t_first_token":348662.125588454,"t_first_token_unix":1779779527.1395319,"t_done":348664.365935815,"t_done_unix":1779779529.3798792},{"request_id":"1311753:2:1337278:426","session_id":"1311753","input_length":36518,"t_proxy_recv":348656.046338653,"t_decision_unix":1779779521.0602798,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":4,"chosen_idx":0,"affinity_cache_hit":19968,"affinity_cache_ratio":0.5467988389287475,"affinity_num_requests":3,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":36518,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":893,"cache_hit":0,"new_prefill":36518,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":0,"new_prefill":36518,"score_linear":62618.0,"score_lmetric":36518},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":12644,"ongoing_decode_tokens":0,"pending_prefill_tokens":12644,"num_requests":1,"active_p_offloads":0,"cached_blocks":1214,"cache_hit":0,"new_prefill":36518,"score_linear":12644.0,"score_lmetric":49162},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1007,"cache_hit":0,"new_prefill":36518,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":178114,"ongoing_decode_tokens":55130,"pending_prefill_tokens":12904,"num_requests":3,"active_p_offloads":0,"cached_blocks":1616,"cache_hit":19968,"new_prefill":16550,"score_linear":158146.0,"score_lmetric":88362},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":61229,"ongoing_decode_tokens":0,"pending_prefill_tokens":57645,"num_requests":1,"active_p_offloads":0,"cached_blocks":905,"cache_hit":0,"new_prefill":36518,"score_linear":61229.0,"score_lmetric":94163},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":20009,"ongoing_decode_tokens":0,"pending_prefill_tokens":5673,"num_requests":1,"active_p_offloads":0,"cached_blocks":972,"cache_hit":0,"new_prefill":36518,"score_linear":20009.0,"score_lmetric":42191},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":913,"cache_hit":0,"new_prefill":36518,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348663.931198242,"t_first_token_unix":1779779528.945142,"t_done":348666.448801487,"t_done_unix":1779779531.4627447},{"request_id":"1316848:13:1337519:427","session_id":"1316848","input_length":45005,"t_proxy_recv":348657.024303223,"t_decision_unix":1779779522.038244,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":44544,"affinity_cache_ratio":0.9897566937007,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":44544,"estimated_new_tokens":461,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":36518,"ongoing_decode_tokens":0,"pending_prefill_tokens":36518,"num_requests":1,"active_p_offloads":0,"cached_blocks":893,"cache_hit":44544,"new_prefill":461,"score_linear":-8026.0,"score_lmetric":36979},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":6144,"new_prefill":38861,"score_linear":56474.0,"score_lmetric":38861},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1238,"cache_hit":0,"new_prefill":45005,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1007,"cache_hit":0,"new_prefill":45005,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":178114,"ongoing_decode_tokens":55130,"pending_prefill_tokens":12904,"num_requests":3,"active_p_offloads":0,"cached_blocks":1616,"cache_hit":1024,"new_prefill":43981,"score_linear":177090.0,"score_lmetric":170655},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":61229,"ongoing_decode_tokens":0,"pending_prefill_tokens":57645,"num_requests":1,"active_p_offloads":0,"cached_blocks":905,"cache_hit":0,"new_prefill":45005,"score_linear":61229.0,"score_lmetric":102650},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":20009,"ongoing_decode_tokens":20009,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":972,"cache_hit":0,"new_prefill":45005,"score_linear":20009.0,"score_lmetric":45005},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":913,"cache_hit":0,"new_prefill":45005,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-8026.0,"chosen_score_lmetric":36979,"t_first_token":348663.937027108,"t_first_token_unix":1779779528.9509706,"t_done":348666.617246175,"t_done_unix":1779779531.6311898},{"request_id":"1339977:1:1339977:442","session_id":"1339977","input_length":13816,"t_proxy_recv":348665.725888343,"t_decision_unix":1779779530.7398293,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":13816,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":176906,"ongoing_decode_tokens":81523,"pending_prefill_tokens":8343,"num_requests":3,"active_p_offloads":0,"cached_blocks":893,"cache_hit":0,"new_prefill":13816,"score_linear":176906.0,"score_lmetric":66477},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":0,"new_prefill":13816,"score_linear":62618.0,"score_lmetric":13816},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":45994,"ongoing_decode_tokens":45994,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1238,"cache_hit":0,"new_prefill":13816,"score_linear":45994.0,"score_lmetric":13816},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1026,"cache_hit":0,"new_prefill":13816,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1643,"cache_hit":0,"new_prefill":13816,"score_linear":55130.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":66152,"ongoing_decode_tokens":0,"pending_prefill_tokens":4712,"num_requests":1,"active_p_offloads":0,"cached_blocks":1019,"cache_hit":0,"new_prefill":13816,"score_linear":66152.0,"score_lmetric":18528},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":31527,"ongoing_decode_tokens":31527,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":988,"cache_hit":0,"new_prefill":13816,"score_linear":31527.0,"score_lmetric":27632},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":917,"cache_hit":0,"new_prefill":13816,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348666.920760986,"t_first_token_unix":1779779531.9347045,"t_done":348667.118340335,"t_done_unix":1779779532.132284},{"request_id":"1259178:4:1339295:439","session_id":"1259178","input_length":24462,"t_proxy_recv":348663.144864779,"t_decision_unix":1779779528.158806,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":22528,"affinity_cache_ratio":0.9209385986427929,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":22528,"estimated_new_tokens":1934,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":176906,"ongoing_decode_tokens":0,"pending_prefill_tokens":45322,"num_requests":3,"active_p_offloads":0,"cached_blocks":893,"cache_hit":0,"new_prefill":24462,"score_linear":176906.0,"score_lmetric":209352},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":0,"new_prefill":24462,"score_linear":62618.0,"score_lmetric":24462},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1238,"cache_hit":0,"new_prefill":24462,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":50215,"ongoing_decode_tokens":50215,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1025,"cache_hit":0,"new_prefill":24462,"score_linear":50215.0,"score_lmetric":24462},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1643,"cache_hit":0,"new_prefill":24462,"score_linear":55130.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":61229,"ongoing_decode_tokens":61229,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":906,"cache_hit":0,"new_prefill":24462,"score_linear":61229.0,"score_lmetric":24462},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":988,"cache_hit":22528,"new_prefill":1934,"score_linear":-15463.0,"score_lmetric":1934},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":917,"cache_hit":0,"new_prefill":24462,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-15463.0,"chosen_score_lmetric":1934,"t_first_token":348663.571586626,"t_first_token_unix":1779779528.5855303,"t_done":348668.807653129,"t_done_unix":1779779533.8215969},{"request_id":"1279412:20:1337591:428","session_id":"1279412","input_length":95383,"t_proxy_recv":348657.272383879,"t_decision_unix":1779779522.2863245,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":87040,"affinity_cache_ratio":0.912531583196167,"affinity_num_requests":2,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":87040,"estimated_new_tokens":8343,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":81523,"ongoing_decode_tokens":0,"pending_prefill_tokens":36979,"num_requests":2,"active_p_offloads":0,"cached_blocks":893,"cache_hit":87040,"new_prefill":8343,"score_linear":-5517.0,"score_lmetric":90644},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":0,"new_prefill":95383,"score_linear":62618.0,"score_lmetric":95383},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1238,"cache_hit":0,"new_prefill":95383,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1007,"cache_hit":0,"new_prefill":95383,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":178114,"ongoing_decode_tokens":55130,"pending_prefill_tokens":12904,"num_requests":3,"active_p_offloads":0,"cached_blocks":1616,"cache_hit":0,"new_prefill":95383,"score_linear":178114.0,"score_lmetric":324861},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":61229,"ongoing_decode_tokens":0,"pending_prefill_tokens":57645,"num_requests":1,"active_p_offloads":0,"cached_blocks":905,"cache_hit":0,"new_prefill":95383,"score_linear":61229.0,"score_lmetric":153028},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":983,"cache_hit":12800,"new_prefill":82583,"score_linear":-12800.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":913,"cache_hit":12800,"new_prefill":82583,"score_linear":-12800.0,"score_lmetric":0}],"chosen_score_linear":-5517.0,"chosen_score_lmetric":90644,"t_first_token":348666.009179644,"t_first_token_unix":1779779531.0231235,"t_done":348669.562039711,"t_done_unix":1779779534.5759835},{"request_id":"1340903:1:1340903:446","session_id":"1340903","input_length":2279,"t_proxy_recv":348668.855746719,"t_decision_unix":1779779533.869688,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":2279,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":145071,"ongoing_decode_tokens":145071,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":964,"cache_hit":0,"new_prefill":2279,"score_linear":145071.0,"score_lmetric":4558},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":0,"new_prefill":2279,"score_linear":62618.0,"score_lmetric":2279},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":67215,"ongoing_decode_tokens":45994,"pending_prefill_tokens":21221,"num_requests":2,"active_p_offloads":0,"cached_blocks":1238,"cache_hit":0,"new_prefill":2279,"score_linear":67215.0,"score_lmetric":47000},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1053,"cache_hit":0,"new_prefill":2279,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":103511,"ongoing_decode_tokens":55130,"pending_prefill_tokens":48381,"num_requests":1,"active_p_offloads":0,"cached_blocks":1643,"cache_hit":0,"new_prefill":2279,"score_linear":103511.0,"score_lmetric":50660},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":66152,"ongoing_decode_tokens":66152,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1019,"cache_hit":0,"new_prefill":2279,"score_linear":66152.0,"score_lmetric":2279},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":993,"cache_hit":0,"new_prefill":2279,"score_linear":7065.0,"score_lmetric":2279},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":917,"cache_hit":0,"new_prefill":2279,"score_linear":21371.0,"score_lmetric":2279}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348669.025694709,"t_first_token_unix":1779779534.0396383,"t_done":348670.049936329,"t_done_unix":1779779535.06388},{"request_id":"1341005:1:1341005:447","session_id":"1341005","input_length":5280,"t_proxy_recv":348669.200542848,"t_decision_unix":1779779534.2144837,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":5280,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":5280,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":145071,"ongoing_decode_tokens":145071,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":964,"cache_hit":0,"new_prefill":5280,"score_linear":145071.0,"score_lmetric":10560},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":0,"new_prefill":5280,"score_linear":62618.0,"score_lmetric":5280},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":67215,"ongoing_decode_tokens":67215,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1238,"cache_hit":0,"new_prefill":5280,"score_linear":67215.0,"score_lmetric":10560},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":2279,"ongoing_decode_tokens":2279,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1053,"cache_hit":0,"new_prefill":5280,"score_linear":2279.0,"score_lmetric":5280},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":103511,"ongoing_decode_tokens":55130,"pending_prefill_tokens":48381,"num_requests":1,"active_p_offloads":0,"cached_blocks":1643,"cache_hit":0,"new_prefill":5280,"score_linear":103511.0,"score_lmetric":53661},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":66152,"ongoing_decode_tokens":66152,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1019,"cache_hit":0,"new_prefill":5280,"score_linear":66152.0,"score_lmetric":5280},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":993,"cache_hit":0,"new_prefill":5280,"score_linear":7065.0,"score_lmetric":5280},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":917,"cache_hit":0,"new_prefill":5280,"score_linear":21371.0,"score_lmetric":5280}],"chosen_score_linear":21371.0,"chosen_score_lmetric":5280,"t_first_token":348669.539192531,"t_first_token_unix":1779779534.5531356,"t_done":348670.064795891,"t_done_unix":1779779535.0787392},{"request_id":"1309252:3:1339771:441","session_id":"1309252","input_length":45994,"t_proxy_recv":348664.974075323,"t_decision_unix":1779779529.9880161,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":45568,"affinity_cache_ratio":0.9907379223376962,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":45568,"estimated_new_tokens":426,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":176906,"ongoing_decode_tokens":81523,"pending_prefill_tokens":8343,"num_requests":3,"active_p_offloads":0,"cached_blocks":893,"cache_hit":0,"new_prefill":45994,"score_linear":176906.0,"score_lmetric":163011},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":0,"new_prefill":45994,"score_linear":62618.0,"score_lmetric":45994},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1238,"cache_hit":45568,"new_prefill":426,"score_linear":-45568.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1026,"cache_hit":0,"new_prefill":45994,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1643,"cache_hit":0,"new_prefill":45994,"score_linear":55130.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":66152,"ongoing_decode_tokens":0,"pending_prefill_tokens":4712,"num_requests":1,"active_p_offloads":0,"cached_blocks":1019,"cache_hit":0,"new_prefill":45994,"score_linear":66152.0,"score_lmetric":50706},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":31527,"ongoing_decode_tokens":31527,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":988,"cache_hit":0,"new_prefill":45994,"score_linear":31527.0,"score_lmetric":91988},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":917,"cache_hit":0,"new_prefill":45994,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-45568.0,"chosen_score_lmetric":0,"t_first_token":348665.133211387,"t_first_token_unix":1779779530.147155,"t_done":348671.158709119,"t_done_unix":1779779536.1726525},{"request_id":"1340290:1:1340290:445","session_id":"1340290","input_length":21221,"t_proxy_recv":348666.850117702,"t_decision_unix":1779779531.8640592,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":21221,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":21221,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":145071,"ongoing_decode_tokens":95383,"pending_prefill_tokens":5144,"num_requests":2,"active_p_offloads":0,"cached_blocks":964,"cache_hit":0,"new_prefill":21221,"score_linear":145071.0,"score_lmetric":52730},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":0,"new_prefill":21221,"score_linear":62618.0,"score_lmetric":21221},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":45994,"ongoing_decode_tokens":45994,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1238,"cache_hit":0,"new_prefill":21221,"score_linear":45994.0,"score_lmetric":21221},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":13816,"ongoing_decode_tokens":0,"pending_prefill_tokens":13816,"num_requests":1,"active_p_offloads":0,"cached_blocks":1026,"cache_hit":0,"new_prefill":21221,"score_linear":13816.0,"score_lmetric":35037},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":103511,"ongoing_decode_tokens":55130,"pending_prefill_tokens":48381,"num_requests":1,"active_p_offloads":0,"cached_blocks":1643,"cache_hit":0,"new_prefill":21221,"score_linear":103511.0,"score_lmetric":69602},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":66152,"ongoing_decode_tokens":66152,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1019,"cache_hit":0,"new_prefill":21221,"score_linear":66152.0,"score_lmetric":21221},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":31527,"ongoing_decode_tokens":31527,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":988,"cache_hit":1024,"new_prefill":20197,"score_linear":30503.0,"score_lmetric":40394},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":917,"cache_hit":0,"new_prefill":21221,"score_linear":21371.0,"score_lmetric":21221}],"chosen_score_linear":45994.0,"chosen_score_lmetric":21221,"t_first_token":348669.114554986,"t_first_token_unix":1779779534.1284983,"t_done":348671.609152809,"t_done_unix":1779779536.6230965},{"request_id":"1279412:21:1339581:440","session_id":"1279412","input_length":95543,"t_proxy_recv":348669.576995789,"t_decision_unix":1779779534.5909374,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":95232,"affinity_cache_ratio":0.9967449211349864,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":95232,"estimated_new_tokens":311,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":49688,"ongoing_decode_tokens":49688,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":980,"cache_hit":95232,"new_prefill":311,"score_linear":-45544.0,"score_lmetric":311},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":0,"new_prefill":95543,"score_linear":62618.0,"score_lmetric":95543},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":67215,"ongoing_decode_tokens":67215,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1238,"cache_hit":0,"new_prefill":95543,"score_linear":67215.0,"score_lmetric":191086},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":2279,"ongoing_decode_tokens":2279,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1053,"cache_hit":0,"new_prefill":95543,"score_linear":2279.0,"score_lmetric":95543},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":103511,"ongoing_decode_tokens":55130,"pending_prefill_tokens":48381,"num_requests":1,"active_p_offloads":0,"cached_blocks":1643,"cache_hit":0,"new_prefill":95543,"score_linear":103511.0,"score_lmetric":143924},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":66152,"ongoing_decode_tokens":66152,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1019,"cache_hit":0,"new_prefill":95543,"score_linear":66152.0,"score_lmetric":95543},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":993,"cache_hit":12800,"new_prefill":82743,"score_linear":-5735.0,"score_lmetric":82743},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":26651,"ongoing_decode_tokens":26651,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":917,"cache_hit":12800,"new_prefill":82743,"score_linear":13851.0,"score_lmetric":165486}],"chosen_score_linear":-45544.0,"chosen_score_lmetric":311,"t_first_token":348669.716779727,"t_first_token_unix":1779779534.7307234,"t_done":348672.784385505,"t_done_unix":1779779537.798329},{"request_id":"1341912:1:1341912:449","session_id":"1341912","input_length":9909,"t_proxy_recv":348672.233839784,"t_decision_unix":1779779537.247781,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":9909,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":145231,"ongoing_decode_tokens":145231,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":980,"cache_hit":0,"new_prefill":9909,"score_linear":145231.0,"score_lmetric":19818},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":0,"new_prefill":9909,"score_linear":62618.0,"score_lmetric":9909},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":37527,"ongoing_decode_tokens":0,"pending_prefill_tokens":13975,"num_requests":1,"active_p_offloads":0,"cached_blocks":1281,"cache_hit":0,"new_prefill":9909,"score_linear":37527.0,"score_lmetric":23884},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1057,"cache_hit":0,"new_prefill":9909,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":103511,"ongoing_decode_tokens":55130,"pending_prefill_tokens":48381,"num_requests":1,"active_p_offloads":0,"cached_blocks":1643,"cache_hit":0,"new_prefill":9909,"score_linear":103511.0,"score_lmetric":58290},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":66152,"ongoing_decode_tokens":66152,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1019,"cache_hit":0,"new_prefill":9909,"score_linear":66152.0,"score_lmetric":9909},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":993,"cache_hit":0,"new_prefill":9909,"score_linear":7065.0,"score_lmetric":9909},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":927,"cache_hit":0,"new_prefill":9909,"score_linear":21371.0,"score_lmetric":9909}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348673.014296233,"t_first_token_unix":1779779538.0282397,"t_done":348673.512448394,"t_done_unix":1779779538.5263917},{"request_id":"1342327:1:1342327:452","session_id":"1342327","input_length":6180,"t_proxy_recv":348673.656059556,"t_decision_unix":1779779538.6700013,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":6180,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":49688,"ongoing_decode_tokens":49688,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":981,"cache_hit":0,"new_prefill":6180,"score_linear":49688.0,"score_lmetric":6180},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":0,"new_prefill":6180,"score_linear":62618.0,"score_lmetric":6180},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":58907,"ongoing_decode_tokens":0,"pending_prefill_tokens":14363,"num_requests":2,"active_p_offloads":0,"cached_blocks":1281,"cache_hit":0,"new_prefill":6180,"score_linear":58907.0,"score_lmetric":41086},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1076,"cache_hit":0,"new_prefill":6180,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":103511,"ongoing_decode_tokens":55130,"pending_prefill_tokens":48381,"num_requests":1,"active_p_offloads":0,"cached_blocks":1643,"cache_hit":0,"new_prefill":6180,"score_linear":103511.0,"score_lmetric":54561},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":66152,"ongoing_decode_tokens":66152,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1019,"cache_hit":0,"new_prefill":6180,"score_linear":66152.0,"score_lmetric":6180},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":993,"cache_hit":0,"new_prefill":6180,"score_linear":7065.0,"score_lmetric":6180},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":41258,"ongoing_decode_tokens":21371,"pending_prefill_tokens":19887,"num_requests":2,"active_p_offloads":0,"cached_blocks":927,"cache_hit":0,"new_prefill":6180,"score_linear":41258.0,"score_lmetric":52134}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348674.057448172,"t_first_token_unix":1779779539.071392,"t_done":348674.36052942,"t_done_unix":1779779539.3744726},{"request_id":"1316848:14:1338658:435","session_id":"1316848","input_length":49688,"t_proxy_recv":348666.63203053,"t_decision_unix":1779779531.6459723,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":44544,"affinity_cache_ratio":0.8964739977459346,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":44544,"estimated_new_tokens":5144,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":95383,"ongoing_decode_tokens":95383,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":964,"cache_hit":44544,"new_prefill":5144,"score_linear":50839.0,"score_lmetric":5144},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":6144,"new_prefill":43544,"score_linear":56474.0,"score_lmetric":43544},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":45994,"ongoing_decode_tokens":45994,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1238,"cache_hit":0,"new_prefill":49688,"score_linear":45994.0,"score_lmetric":49688},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":13816,"ongoing_decode_tokens":0,"pending_prefill_tokens":13816,"num_requests":1,"active_p_offloads":0,"cached_blocks":1026,"cache_hit":0,"new_prefill":49688,"score_linear":13816.0,"score_lmetric":63504},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1643,"cache_hit":1024,"new_prefill":48664,"score_linear":54106.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":66152,"ongoing_decode_tokens":66152,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1019,"cache_hit":0,"new_prefill":49688,"score_linear":66152.0,"score_lmetric":49688},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":31527,"ongoing_decode_tokens":31527,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":988,"cache_hit":0,"new_prefill":49688,"score_linear":31527.0,"score_lmetric":99376},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":21371,"ongoing_decode_tokens":0,"pending_prefill_tokens":6523,"num_requests":1,"active_p_offloads":0,"cached_blocks":917,"cache_hit":0,"new_prefill":49688,"score_linear":21371.0,"score_lmetric":56211}],"chosen_score_linear":50839.0,"chosen_score_lmetric":5144,"t_first_token":348668.200198411,"t_first_token_unix":1779779533.2141423,"t_done":348674.411025129,"t_done_unix":1779779539.4249687},{"request_id":"1341948:1:1341948:450","session_id":"1341948","input_length":19887,"t_proxy_recv":348672.354762425,"t_decision_unix":1779779537.368704,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":19887,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":19887,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":145231,"ongoing_decode_tokens":145231,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":980,"cache_hit":0,"new_prefill":19887,"score_linear":145231.0,"score_lmetric":39774},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":0,"new_prefill":19887,"score_linear":62618.0,"score_lmetric":19887},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":37527,"ongoing_decode_tokens":0,"pending_prefill_tokens":13975,"num_requests":1,"active_p_offloads":0,"cached_blocks":1281,"cache_hit":0,"new_prefill":19887,"score_linear":37527.0,"score_lmetric":33862},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":9909,"ongoing_decode_tokens":0,"pending_prefill_tokens":9909,"num_requests":1,"active_p_offloads":0,"cached_blocks":1057,"cache_hit":0,"new_prefill":19887,"score_linear":9909.0,"score_lmetric":29796},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":103511,"ongoing_decode_tokens":55130,"pending_prefill_tokens":48381,"num_requests":1,"active_p_offloads":0,"cached_blocks":1643,"cache_hit":0,"new_prefill":19887,"score_linear":103511.0,"score_lmetric":68268},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":66152,"ongoing_decode_tokens":66152,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1019,"cache_hit":0,"new_prefill":19887,"score_linear":66152.0,"score_lmetric":19887},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":993,"cache_hit":0,"new_prefill":19887,"score_linear":7065.0,"score_lmetric":19887},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":927,"cache_hit":0,"new_prefill":19887,"score_linear":21371.0,"score_lmetric":19887}],"chosen_score_linear":21371.0,"chosen_score_lmetric":19887,"t_first_token":348674.403883573,"t_first_token_unix":1779779539.4178274,"t_done":348674.752802775,"t_done_unix":1779779539.766746},{"request_id":"1340903:2:1342632:453","session_id":"1340903","input_length":2129,"t_proxy_recv":348674.692561151,"t_decision_unix":1779779539.7065027,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":2048,"affinity_cache_ratio":0.9619539689995303,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":2048,"estimated_new_tokens":81,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":992,"cache_hit":0,"new_prefill":2129,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":0,"new_prefill":2129,"score_linear":62618.0,"score_lmetric":2129},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":58907,"ongoing_decode_tokens":0,"pending_prefill_tokens":14363,"num_requests":2,"active_p_offloads":0,"cached_blocks":1281,"cache_hit":0,"new_prefill":2129,"score_linear":58907.0,"score_lmetric":32984},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1088,"cache_hit":2048,"new_prefill":81,"score_linear":-2048.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":103511,"ongoing_decode_tokens":55130,"pending_prefill_tokens":48381,"num_requests":1,"active_p_offloads":0,"cached_blocks":1643,"cache_hit":0,"new_prefill":2129,"score_linear":103511.0,"score_lmetric":50510},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":66152,"ongoing_decode_tokens":66152,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1019,"cache_hit":0,"new_prefill":2129,"score_linear":66152.0,"score_lmetric":2129},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":993,"cache_hit":0,"new_prefill":2129,"score_linear":7065.0,"score_lmetric":2129},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":41258,"ongoing_decode_tokens":41258,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":927,"cache_hit":0,"new_prefill":2129,"score_linear":41258.0,"score_lmetric":4258}],"chosen_score_linear":-2048.0,"chosen_score_lmetric":0,"t_first_token":348674.706817394,"t_first_token_unix":1779779539.7207606,"t_done":348674.780260845,"t_done_unix":1779779539.7942042},{"request_id":"1340290:2:1342119:451","session_id":"1340290","input_length":21380,"t_proxy_recv":348672.889924989,"t_decision_unix":1779779537.9038656,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":20992,"affinity_cache_ratio":0.9818521983161833,"affinity_num_requests":1,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":20992,"estimated_new_tokens":388,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":49688,"ongoing_decode_tokens":49688,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":981,"cache_hit":0,"new_prefill":21380,"score_linear":49688.0,"score_lmetric":21380},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":0,"new_prefill":21380,"score_linear":62618.0,"score_lmetric":21380},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":37527,"ongoing_decode_tokens":0,"pending_prefill_tokens":13975,"num_requests":1,"active_p_offloads":0,"cached_blocks":1281,"cache_hit":20992,"new_prefill":388,"score_linear":16535.0,"score_lmetric":14363},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":9909,"ongoing_decode_tokens":0,"pending_prefill_tokens":9909,"num_requests":1,"active_p_offloads":0,"cached_blocks":1057,"cache_hit":0,"new_prefill":21380,"score_linear":9909.0,"score_lmetric":31289},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":103511,"ongoing_decode_tokens":55130,"pending_prefill_tokens":48381,"num_requests":1,"active_p_offloads":0,"cached_blocks":1643,"cache_hit":0,"new_prefill":21380,"score_linear":103511.0,"score_lmetric":69761},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":66152,"ongoing_decode_tokens":66152,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1019,"cache_hit":0,"new_prefill":21380,"score_linear":66152.0,"score_lmetric":21380},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":993,"cache_hit":1024,"new_prefill":20356,"score_linear":6041.0,"score_lmetric":20356},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":41258,"ongoing_decode_tokens":21371,"pending_prefill_tokens":19887,"num_requests":2,"active_p_offloads":0,"cached_blocks":927,"cache_hit":0,"new_prefill":21380,"score_linear":41258.0,"score_lmetric":82534}],"chosen_score_linear":16535.0,"chosen_score_lmetric":14363,"t_first_token":348674.839552439,"t_first_token_unix":1779779539.8535025,"t_done":348675.810518418,"t_done_unix":1779779540.8244622},{"request_id":"1340278:1:1340278:444","session_id":"1340278","input_length":48381,"t_proxy_recv":348666.826056483,"t_decision_unix":1779779531.8399978,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":48381,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":145071,"ongoing_decode_tokens":95383,"pending_prefill_tokens":5144,"num_requests":2,"active_p_offloads":0,"cached_blocks":964,"cache_hit":0,"new_prefill":48381,"score_linear":145071.0,"score_lmetric":107050},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":0,"new_prefill":48381,"score_linear":62618.0,"score_lmetric":48381},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":45994,"ongoing_decode_tokens":45994,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1238,"cache_hit":0,"new_prefill":48381,"score_linear":45994.0,"score_lmetric":48381},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":13816,"ongoing_decode_tokens":0,"pending_prefill_tokens":13816,"num_requests":1,"active_p_offloads":0,"cached_blocks":1026,"cache_hit":0,"new_prefill":48381,"score_linear":13816.0,"score_lmetric":62197},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1643,"cache_hit":0,"new_prefill":48381,"score_linear":55130.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":66152,"ongoing_decode_tokens":66152,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1019,"cache_hit":0,"new_prefill":48381,"score_linear":66152.0,"score_lmetric":48381},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":31527,"ongoing_decode_tokens":31527,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":988,"cache_hit":0,"new_prefill":48381,"score_linear":31527.0,"score_lmetric":96762},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":917,"cache_hit":0,"new_prefill":48381,"score_linear":21371.0,"score_lmetric":48381}],"chosen_score_linear":55130.0,"chosen_score_lmetric":0,"t_first_token":348675.667032235,"t_first_token_unix":1779779540.6809766,"t_done":348676.580459762,"t_done_unix":1779779541.594403},{"request_id":"1341005:2:1342961:456","session_id":"1341005","input_length":9360,"t_proxy_recv":348675.859623916,"t_decision_unix":1779779540.8735657,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":5120,"affinity_cache_ratio":0.5470085470085471,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":5120,"estimated_new_tokens":4240,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":59219,"ongoing_decode_tokens":0,"pending_prefill_tokens":59219,"num_requests":1,"active_p_offloads":0,"cached_blocks":992,"cache_hit":0,"new_prefill":9360,"score_linear":59219.0,"score_lmetric":68579},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":0,"new_prefill":9360,"score_linear":62618.0,"score_lmetric":9360},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":37527,"ongoing_decode_tokens":37527,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1281,"cache_hit":0,"new_prefill":9360,"score_linear":37527.0,"score_lmetric":9360},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":65289,"ongoing_decode_tokens":0,"pending_prefill_tokens":65289,"num_requests":1,"active_p_offloads":0,"cached_blocks":1088,"cache_hit":0,"new_prefill":9360,"score_linear":65289.0,"score_lmetric":74649},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":103511,"ongoing_decode_tokens":103511,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1643,"cache_hit":0,"new_prefill":9360,"score_linear":103511.0,"score_lmetric":9360},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":66152,"ongoing_decode_tokens":66152,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1019,"cache_hit":0,"new_prefill":9360,"score_linear":66152.0,"score_lmetric":9360},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":993,"cache_hit":0,"new_prefill":9360,"score_linear":7065.0,"score_lmetric":9360},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":965,"cache_hit":5120,"new_prefill":4240,"score_linear":16251.0,"score_lmetric":4240}],"chosen_score_linear":16251.0,"chosen_score_lmetric":4240,"t_first_token":348676.24605949,"t_first_token_unix":1779779541.2600033,"t_done":348676.662479449,"t_done_unix":1779779541.6764228},{"request_id":"1334223:2:1341796:448","session_id":"1334223","input_length":37527,"t_proxy_recv":348671.804696377,"t_decision_unix":1779779536.8186378,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":23552,"affinity_cache_ratio":0.6276014602819303,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":23552,"estimated_new_tokens":13975,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":145231,"ongoing_decode_tokens":145231,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":980,"cache_hit":0,"new_prefill":37527,"score_linear":145231.0,"score_lmetric":75054},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":0,"new_prefill":37527,"score_linear":62618.0,"score_lmetric":37527},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1281,"cache_hit":23552,"new_prefill":13975,"score_linear":-23552.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1057,"cache_hit":0,"new_prefill":37527,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":103511,"ongoing_decode_tokens":55130,"pending_prefill_tokens":48381,"num_requests":1,"active_p_offloads":0,"cached_blocks":1643,"cache_hit":0,"new_prefill":37527,"score_linear":103511.0,"score_lmetric":85908},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":66152,"ongoing_decode_tokens":66152,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1019,"cache_hit":0,"new_prefill":37527,"score_linear":66152.0,"score_lmetric":37527},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":993,"cache_hit":0,"new_prefill":37527,"score_linear":7065.0,"score_lmetric":37527},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":927,"cache_hit":0,"new_prefill":37527,"score_linear":21371.0,"score_lmetric":37527}],"chosen_score_linear":-23552.0,"chosen_score_lmetric":0,"t_first_token":348674.827284699,"t_first_token_unix":1779779539.841235,"t_done":348676.928563571,"t_done_unix":1779779541.942507},{"request_id":"1343191:1:1343191:457","session_id":"1343191","input_length":11912,"t_proxy_recv":348676.765402735,"t_decision_unix":1779779541.779344,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":11912,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":59219,"ongoing_decode_tokens":0,"pending_prefill_tokens":59219,"num_requests":1,"active_p_offloads":0,"cached_blocks":992,"cache_hit":0,"new_prefill":11912,"score_linear":59219.0,"score_lmetric":71131},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":0,"new_prefill":11912,"score_linear":62618.0,"score_lmetric":11912},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":37527,"ongoing_decode_tokens":37527,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1281,"cache_hit":0,"new_prefill":11912,"score_linear":37527.0,"score_lmetric":11912},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":65289,"ongoing_decode_tokens":0,"pending_prefill_tokens":65289,"num_requests":1,"active_p_offloads":0,"cached_blocks":1088,"cache_hit":0,"new_prefill":11912,"score_linear":65289.0,"score_lmetric":77201},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1737,"cache_hit":0,"new_prefill":11912,"score_linear":55130.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":66152,"ongoing_decode_tokens":66152,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1019,"cache_hit":0,"new_prefill":11912,"score_linear":66152.0,"score_lmetric":11912},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":993,"cache_hit":0,"new_prefill":11912,"score_linear":7065.0,"score_lmetric":11912},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":973,"cache_hit":0,"new_prefill":11912,"score_linear":21371.0,"score_lmetric":11912}],"chosen_score_linear":55130.0,"chosen_score_lmetric":0,"t_first_token":348677.751147973,"t_first_token_unix":1779779542.7650976,"t_done":348677.852913322,"t_done_unix":1779779542.8668568},{"request_id":"1314357:3:1338999:437","session_id":"1314357","input_length":66152,"t_proxy_recv":348664.377336621,"t_decision_unix":1779779529.391278,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":61440,"affinity_cache_ratio":0.9287701052122385,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":61440,"estimated_new_tokens":4712,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":176906,"ongoing_decode_tokens":81523,"pending_prefill_tokens":8343,"num_requests":3,"active_p_offloads":0,"cached_blocks":893,"cache_hit":0,"new_prefill":66152,"score_linear":176906.0,"score_lmetric":223485},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":0,"new_prefill":66152,"score_linear":62618.0,"score_lmetric":66152},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1238,"cache_hit":0,"new_prefill":66152,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1026,"cache_hit":0,"new_prefill":66152,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1643,"cache_hit":56832,"new_prefill":9320,"score_linear":-1702.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1019,"cache_hit":61440,"new_prefill":4712,"score_linear":-61440.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":31527,"ongoing_decode_tokens":31527,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":988,"cache_hit":0,"new_prefill":66152,"score_linear":31527.0,"score_lmetric":132304},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":917,"cache_hit":0,"new_prefill":66152,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-61440.0,"chosen_score_lmetric":0,"t_first_token":348666.412913107,"t_first_token_unix":1779779531.4268568,"t_done":348678.667917092,"t_done_unix":1779779543.6818607},{"request_id":"1343312:1:1343312:459","session_id":"1343312","input_length":21402,"t_proxy_recv":348677.130061715,"t_decision_unix":1779779542.1440032,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":21402,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":59219,"ongoing_decode_tokens":0,"pending_prefill_tokens":59219,"num_requests":1,"active_p_offloads":0,"cached_blocks":992,"cache_hit":0,"new_prefill":21402,"score_linear":59219.0,"score_lmetric":80621},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":0,"new_prefill":21402,"score_linear":62618.0,"score_lmetric":21402},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1308,"cache_hit":0,"new_prefill":21402,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":75521,"ongoing_decode_tokens":0,"pending_prefill_tokens":65793,"num_requests":2,"active_p_offloads":0,"cached_blocks":1088,"cache_hit":0,"new_prefill":21402,"score_linear":75521.0,"score_lmetric":174390},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":67042,"ongoing_decode_tokens":55130,"pending_prefill_tokens":11912,"num_requests":1,"active_p_offloads":0,"cached_blocks":1737,"cache_hit":0,"new_prefill":21402,"score_linear":67042.0,"score_lmetric":33314},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":66152,"ongoing_decode_tokens":66152,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1019,"cache_hit":0,"new_prefill":21402,"score_linear":66152.0,"score_lmetric":21402},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":993,"cache_hit":0,"new_prefill":21402,"score_linear":7065.0,"score_lmetric":21402},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":973,"cache_hit":0,"new_prefill":21402,"score_linear":21371.0,"score_lmetric":21402}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348679.420598905,"t_first_token_unix":1779779544.4345424,"t_done":348679.481489524,"t_done_unix":1779779544.4954326},{"request_id":"1342327:2:1343857:461","session_id":"1342327","input_length":9820,"t_proxy_recv":348678.793133962,"t_decision_unix":1779779543.807075,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":3,"chosen_idx":4,"affinity_cache_hit":6144,"affinity_cache_ratio":0.6256619144602852,"affinity_num_requests":3,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":9820,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":59219,"ongoing_decode_tokens":0,"pending_prefill_tokens":59219,"num_requests":1,"active_p_offloads":0,"cached_blocks":992,"cache_hit":0,"new_prefill":9820,"score_linear":59219.0,"score_lmetric":69039},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":0,"new_prefill":9820,"score_linear":62618.0,"score_lmetric":9820},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":21402,"ongoing_decode_tokens":0,"pending_prefill_tokens":21402,"num_requests":1,"active_p_offloads":0,"cached_blocks":1308,"cache_hit":0,"new_prefill":9820,"score_linear":21402.0,"score_lmetric":31222},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":89624,"ongoing_decode_tokens":0,"pending_prefill_tokens":70680,"num_requests":3,"active_p_offloads":0,"cached_blocks":1088,"cache_hit":6144,"new_prefill":3676,"score_linear":83480.0,"score_lmetric":223068},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1760,"cache_hit":0,"new_prefill":9820,"score_linear":55130.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1029,"cache_hit":0,"new_prefill":9820,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":993,"cache_hit":0,"new_prefill":9820,"score_linear":7065.0,"score_lmetric":9820},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":973,"cache_hit":0,"new_prefill":9820,"score_linear":21371.0,"score_lmetric":9820}],"chosen_score_linear":55130.0,"chosen_score_lmetric":0,"t_first_token":348679.563396384,"t_first_token_unix":1779779544.57734,"t_done":348679.971152148,"t_done_unix":1779779544.9850953},{"request_id":"1342327:3:1344788:466","session_id":"1342327","input_length":12398,"t_proxy_recv":348682.035395135,"t_decision_unix":1779779547.049337,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":9728,"affinity_cache_ratio":0.78464268430392,"affinity_num_requests":0,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":9728,"estimated_new_tokens":2670,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":205447,"ongoing_decode_tokens":0,"pending_prefill_tokens":59527,"num_requests":3,"active_p_offloads":0,"cached_blocks":992,"cache_hit":0,"new_prefill":12398,"score_linear":205447.0,"score_lmetric":215775},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":0,"new_prefill":12398,"score_linear":62618.0,"score_lmetric":12398},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1349,"cache_hit":0,"new_prefill":12398,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":89624,"ongoing_decode_tokens":0,"pending_prefill_tokens":70680,"num_requests":3,"active_p_offloads":0,"cached_blocks":1088,"cache_hit":6144,"new_prefill":6254,"score_linear":83480.0,"score_lmetric":230802},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1779,"cache_hit":9728,"new_prefill":2670,"score_linear":45402.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":32816,"ongoing_decode_tokens":0,"pending_prefill_tokens":32816,"num_requests":1,"active_p_offloads":0,"cached_blocks":1029,"cache_hit":0,"new_prefill":12398,"score_linear":32816.0,"score_lmetric":45214},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":993,"cache_hit":0,"new_prefill":12398,"score_linear":7065.0,"score_lmetric":12398},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":973,"cache_hit":0,"new_prefill":12398,"score_linear":21371.0,"score_lmetric":12398}],"chosen_score_linear":45402.0,"chosen_score_lmetric":0,"t_first_token":348682.368462994,"t_first_token_unix":1779779547.382407,"t_done":348682.62142398,"t_done_unix":1779779547.6353672},{"request_id":"1345921:1:1345921:469","session_id":"1345921","input_length":505,"t_proxy_recv":348685.89521867,"t_decision_unix":1779779550.90916,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":505,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":205447,"ongoing_decode_tokens":0,"pending_prefill_tokens":59527,"num_requests":3,"active_p_offloads":0,"cached_blocks":992,"cache_hit":0,"new_prefill":505,"score_linear":205447.0,"score_lmetric":180096},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":0,"new_prefill":505,"score_linear":62618.0,"score_lmetric":505},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1349,"cache_hit":0,"new_prefill":505,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":89624,"ongoing_decode_tokens":0,"pending_prefill_tokens":70680,"num_requests":3,"active_p_offloads":0,"cached_blocks":1088,"cache_hit":0,"new_prefill":505,"score_linear":89624.0,"score_lmetric":213555},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1784,"cache_hit":0,"new_prefill":505,"score_linear":55130.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":32816,"ongoing_decode_tokens":0,"pending_prefill_tokens":32816,"num_requests":1,"active_p_offloads":0,"cached_blocks":1029,"cache_hit":0,"new_prefill":505,"score_linear":32816.0,"score_lmetric":33321},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":993,"cache_hit":0,"new_prefill":505,"score_linear":7065.0,"score_lmetric":505},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":95317,"ongoing_decode_tokens":95317,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":973,"cache_hit":0,"new_prefill":505,"score_linear":95317.0,"score_lmetric":1010}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348685.948144725,"t_first_token_unix":1779779550.962088,"t_done":348686.276925886,"t_done_unix":1779779551.290869},{"request_id":"1344773:1:1344773:465","session_id":"1344773","input_length":32816,"t_proxy_recv":348681.999561899,"t_decision_unix":1779779547.013503,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":32816,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":205447,"ongoing_decode_tokens":0,"pending_prefill_tokens":59527,"num_requests":3,"active_p_offloads":0,"cached_blocks":992,"cache_hit":0,"new_prefill":32816,"score_linear":205447.0,"score_lmetric":277029},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":0,"new_prefill":32816,"score_linear":62618.0,"score_lmetric":32816},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1349,"cache_hit":0,"new_prefill":32816,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":89624,"ongoing_decode_tokens":0,"pending_prefill_tokens":70680,"num_requests":3,"active_p_offloads":0,"cached_blocks":1088,"cache_hit":0,"new_prefill":32816,"score_linear":89624.0,"score_lmetric":310488},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1779,"cache_hit":0,"new_prefill":32816,"score_linear":55130.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1029,"cache_hit":0,"new_prefill":32816,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":993,"cache_hit":0,"new_prefill":32816,"score_linear":7065.0,"score_lmetric":32816},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":973,"cache_hit":0,"new_prefill":32816,"score_linear":21371.0,"score_lmetric":32816}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348686.533723103,"t_first_token_unix":1779779551.547667,"t_done":348686.677385095,"t_done_unix":1779779551.6913283},{"request_id":"1342327:4:1346218:471","session_id":"1342327","input_length":14842,"t_proxy_recv":348686.997275641,"t_decision_unix":1779779552.0112169,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":12288,"affinity_cache_ratio":0.8279207653954993,"affinity_num_requests":0,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":12288,"estimated_new_tokens":2554,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":205447,"ongoing_decode_tokens":0,"pending_prefill_tokens":59527,"num_requests":3,"active_p_offloads":0,"cached_blocks":992,"cache_hit":0,"new_prefill":14842,"score_linear":205447.0,"score_lmetric":223107},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":0,"new_prefill":14842,"score_linear":62618.0,"score_lmetric":14842},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1350,"cache_hit":0,"new_prefill":14842,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":89624,"ongoing_decode_tokens":0,"pending_prefill_tokens":70680,"num_requests":3,"active_p_offloads":0,"cached_blocks":1088,"cache_hit":6144,"new_prefill":8698,"score_linear":83480.0,"score_lmetric":238134},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1784,"cache_hit":12288,"new_prefill":2554,"score_linear":42842.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1093,"cache_hit":0,"new_prefill":14842,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":993,"cache_hit":0,"new_prefill":14842,"score_linear":7065.0,"score_lmetric":14842},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":95317,"ongoing_decode_tokens":95317,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":973,"cache_hit":0,"new_prefill":14842,"score_linear":95317.0,"score_lmetric":29684}],"chosen_score_linear":42842.0,"chosen_score_lmetric":0,"t_first_token":348687.338311329,"t_first_token_unix":1779779552.3522556,"t_done":348687.428157341,"t_done_unix":1779779552.4421005},{"request_id":"1344773:2:1346419:473","session_id":"1344773","input_length":33008,"t_proxy_recv":348687.766117714,"t_decision_unix":1779779552.780059,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":32768,"affinity_cache_ratio":0.9927290353853612,"affinity_num_requests":0,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":32768,"estimated_new_tokens":240,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":205447,"ongoing_decode_tokens":205447,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":992,"cache_hit":0,"new_prefill":33008,"score_linear":205447.0,"score_lmetric":99024},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":99637,"ongoing_decode_tokens":62618,"pending_prefill_tokens":2715,"num_requests":2,"active_p_offloads":0,"cached_blocks":963,"cache_hit":0,"new_prefill":33008,"score_linear":99637.0,"score_lmetric":71446},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1350,"cache_hit":0,"new_prefill":33008,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":89624,"ongoing_decode_tokens":0,"pending_prefill_tokens":70680,"num_requests":3,"active_p_offloads":0,"cached_blocks":1088,"cache_hit":0,"new_prefill":33008,"score_linear":89624.0,"score_lmetric":311064},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1789,"cache_hit":0,"new_prefill":33008,"score_linear":55130.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1093,"cache_hit":32768,"new_prefill":240,"score_linear":-32768.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":993,"cache_hit":0,"new_prefill":33008,"score_linear":7065.0,"score_lmetric":33008},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":95317,"ongoing_decode_tokens":95317,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":973,"cache_hit":0,"new_prefill":33008,"score_linear":95317.0,"score_lmetric":66016}],"chosen_score_linear":-32768.0,"chosen_score_lmetric":0,"t_first_token":348687.887479963,"t_first_token_unix":1779779552.9014237,"t_done":348688.065211874,"t_done_unix":1779779553.079155},{"request_id":"1316848:15:1344261:463","session_id":"1316848","input_length":50447,"t_proxy_recv":348680.203172849,"t_decision_unix":1779779545.2171144,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":50176,"affinity_cache_ratio":0.9946280254524551,"affinity_num_requests":2,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":50176,"estimated_new_tokens":271,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":155000,"ongoing_decode_tokens":0,"pending_prefill_tokens":59256,"num_requests":2,"active_p_offloads":0,"cached_blocks":992,"cache_hit":50176,"new_prefill":271,"score_linear":104824.0,"score_lmetric":119054},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":6144,"new_prefill":44303,"score_linear":56474.0,"score_lmetric":44303},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1349,"cache_hit":0,"new_prefill":50447,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":89624,"ongoing_decode_tokens":0,"pending_prefill_tokens":70680,"num_requests":3,"active_p_offloads":0,"cached_blocks":1088,"cache_hit":0,"new_prefill":50447,"score_linear":89624.0,"score_lmetric":363381},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1779,"cache_hit":1024,"new_prefill":49423,"score_linear":54106.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1029,"cache_hit":0,"new_prefill":50447,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":993,"cache_hit":0,"new_prefill":50447,"score_linear":7065.0,"score_lmetric":50447},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":973,"cache_hit":0,"new_prefill":50447,"score_linear":21371.0,"score_lmetric":50447}],"chosen_score_linear":104824.0,"chosen_score_lmetric":119054,"t_first_token":348687.61526526,"t_first_token_unix":1779779552.6292083,"t_done":348688.693652283,"t_done_unix":1779779553.7075958},{"request_id":"1269373:18:1345570:467","session_id":"1269373","input_length":73946,"t_proxy_recv":348684.569096994,"t_decision_unix":1779779549.583038,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":73216,"affinity_cache_ratio":0.9901279311930328,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":73216,"estimated_new_tokens":730,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":205447,"ongoing_decode_tokens":0,"pending_prefill_tokens":59527,"num_requests":3,"active_p_offloads":0,"cached_blocks":992,"cache_hit":7680,"new_prefill":66266,"score_linear":197767.0,"score_lmetric":377379},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":0,"new_prefill":73946,"score_linear":62618.0,"score_lmetric":73946},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1349,"cache_hit":0,"new_prefill":73946,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":89624,"ongoing_decode_tokens":0,"pending_prefill_tokens":70680,"num_requests":3,"active_p_offloads":0,"cached_blocks":1088,"cache_hit":0,"new_prefill":73946,"score_linear":89624.0,"score_lmetric":433878},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1784,"cache_hit":7680,"new_prefill":66266,"score_linear":47450.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":32816,"ongoing_decode_tokens":0,"pending_prefill_tokens":32816,"num_requests":1,"active_p_offloads":0,"cached_blocks":1029,"cache_hit":0,"new_prefill":73946,"score_linear":32816.0,"score_lmetric":106762},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":993,"cache_hit":7680,"new_prefill":66266,"score_linear":-615.0,"score_lmetric":66266},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":973,"cache_hit":73216,"new_prefill":730,"score_linear":-51845.0,"score_lmetric":730}],"chosen_score_linear":-51845.0,"chosen_score_lmetric":730,"t_first_token":348685.003343136,"t_first_token_unix":1779779550.017287,"t_done":348689.117127726,"t_done_unix":1779779554.1310713},{"request_id":"1345921:2:1347089:474","session_id":"1345921","input_length":773,"t_proxy_recv":348690.08056482,"t_decision_unix":1779779555.0945063,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":512,"affinity_cache_ratio":0.6623544631306598,"affinity_num_requests":0,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":512,"estimated_new_tokens":261,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":213815,"ongoing_decode_tokens":155000,"pending_prefill_tokens":8639,"num_requests":3,"active_p_offloads":0,"cached_blocks":992,"cache_hit":0,"new_prefill":773,"score_linear":213815.0,"score_lmetric":28236},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":99637,"ongoing_decode_tokens":99637,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":963,"cache_hit":0,"new_prefill":773,"score_linear":99637.0,"score_lmetric":1546},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1350,"cache_hit":512,"new_prefill":261,"score_linear":-512.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":89624,"ongoing_decode_tokens":0,"pending_prefill_tokens":70680,"num_requests":3,"active_p_offloads":0,"cached_blocks":1088,"cache_hit":0,"new_prefill":773,"score_linear":89624.0,"score_lmetric":214359},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1789,"cache_hit":0,"new_prefill":773,"score_linear":55130.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1093,"cache_hit":0,"new_prefill":773,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":993,"cache_hit":0,"new_prefill":773,"score_linear":7065.0,"score_lmetric":773},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":975,"cache_hit":0,"new_prefill":773,"score_linear":21371.0,"score_lmetric":773}],"chosen_score_linear":-512.0,"chosen_score_lmetric":0,"t_first_token":348690.114458127,"t_first_token_unix":1779779555.128401,"t_done":348690.368620272,"t_done_unix":1779779555.382564},{"request_id":"1268861:6:1346340:472","session_id":"1268861","input_length":37019,"t_proxy_recv":348687.439806512,"t_decision_unix":1779779552.4537477,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":34304,"affinity_cache_ratio":0.9266592830708555,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":34304,"estimated_new_tokens":2715,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":205447,"ongoing_decode_tokens":0,"pending_prefill_tokens":59527,"num_requests":3,"active_p_offloads":0,"cached_blocks":992,"cache_hit":0,"new_prefill":37019,"score_linear":205447.0,"score_lmetric":289638},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":34304,"new_prefill":2715,"score_linear":28314.0,"score_lmetric":2715},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1350,"cache_hit":0,"new_prefill":37019,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":89624,"ongoing_decode_tokens":0,"pending_prefill_tokens":70680,"num_requests":3,"active_p_offloads":0,"cached_blocks":1088,"cache_hit":0,"new_prefill":37019,"score_linear":89624.0,"score_lmetric":323097},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1789,"cache_hit":0,"new_prefill":37019,"score_linear":55130.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1093,"cache_hit":0,"new_prefill":37019,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":993,"cache_hit":0,"new_prefill":37019,"score_linear":7065.0,"score_lmetric":37019},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":95317,"ongoing_decode_tokens":95317,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":973,"cache_hit":12288,"new_prefill":24731,"score_linear":83029.0,"score_lmetric":49462}],"chosen_score_linear":28314.0,"chosen_score_lmetric":2715,"t_first_token":348688.10391723,"t_first_token_unix":1779779553.1178606,"t_done":348691.131748945,"t_done_unix":1779779556.1456923},{"request_id":"1338292:2:1343599:460","session_id":"1338292","input_length":14103,"t_proxy_recv":348677.976463264,"t_decision_unix":1779779542.9904046,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":9216,"affinity_cache_ratio":0.6534779834077856,"affinity_num_requests":2,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":9216,"estimated_new_tokens":4887,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":59219,"ongoing_decode_tokens":0,"pending_prefill_tokens":59219,"num_requests":1,"active_p_offloads":0,"cached_blocks":992,"cache_hit":0,"new_prefill":14103,"score_linear":59219.0,"score_lmetric":73322},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":0,"new_prefill":14103,"score_linear":62618.0,"score_lmetric":14103},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":21402,"ongoing_decode_tokens":0,"pending_prefill_tokens":21402,"num_requests":1,"active_p_offloads":0,"cached_blocks":1308,"cache_hit":0,"new_prefill":14103,"score_linear":21402.0,"score_lmetric":35505},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":75521,"ongoing_decode_tokens":0,"pending_prefill_tokens":65793,"num_requests":2,"active_p_offloads":0,"cached_blocks":1088,"cache_hit":9216,"new_prefill":4887,"score_linear":66305.0,"score_lmetric":141360},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1760,"cache_hit":0,"new_prefill":14103,"score_linear":55130.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":66152,"ongoing_decode_tokens":66152,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1019,"cache_hit":0,"new_prefill":14103,"score_linear":66152.0,"score_lmetric":14103},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":993,"cache_hit":0,"new_prefill":14103,"score_linear":7065.0,"score_lmetric":14103},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":973,"cache_hit":0,"new_prefill":14103,"score_linear":21371.0,"score_lmetric":14103}],"chosen_score_linear":66305.0,"chosen_score_lmetric":141360,"t_first_token":348691.23157716,"t_first_token_unix":1779779556.2455208,"t_done":348691.34699738,"t_done_unix":1779779556.3609405},{"request_id":"1339070:1:1339070:438","session_id":"1339070","input_length":7065,"t_proxy_recv":348662.316094565,"t_decision_unix":1779779527.330036,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":7065,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":176906,"ongoing_decode_tokens":0,"pending_prefill_tokens":45322,"num_requests":3,"active_p_offloads":0,"cached_blocks":893,"cache_hit":0,"new_prefill":7065,"score_linear":176906.0,"score_lmetric":157161},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":0,"new_prefill":7065,"score_linear":62618.0,"score_lmetric":7065},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1238,"cache_hit":0,"new_prefill":7065,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":50215,"ongoing_decode_tokens":50215,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1025,"cache_hit":0,"new_prefill":7065,"score_linear":50215.0,"score_lmetric":7065},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1643,"cache_hit":0,"new_prefill":7065,"score_linear":55130.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":144317,"ongoing_decode_tokens":144317,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":905,"cache_hit":0,"new_prefill":7065,"score_linear":144317.0,"score_lmetric":14130},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":988,"cache_hit":0,"new_prefill":7065,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":73073,"ongoing_decode_tokens":73073,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":914,"cache_hit":0,"new_prefill":7065,"score_linear":73073.0,"score_lmetric":7065}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348662.796549932,"t_first_token_unix":1779779527.8104937,"t_done":348691.894971036,"t_done_unix":1779779556.908914},{"request_id":"1316848:16:1345678:468","session_id":"1316848","input_length":58815,"t_proxy_recv":348688.709848583,"t_decision_unix":1779779553.72379,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":50176,"affinity_cache_ratio":0.8531157017767577,"affinity_num_requests":2,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":50176,"estimated_new_tokens":8639,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":155000,"ongoing_decode_tokens":155000,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":992,"cache_hit":50176,"new_prefill":8639,"score_linear":104824.0,"score_lmetric":17278},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":99637,"ongoing_decode_tokens":99637,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":963,"cache_hit":6144,"new_prefill":52671,"score_linear":93493.0,"score_lmetric":105342},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1350,"cache_hit":0,"new_prefill":58815,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":89624,"ongoing_decode_tokens":0,"pending_prefill_tokens":70680,"num_requests":3,"active_p_offloads":0,"cached_blocks":1088,"cache_hit":0,"new_prefill":58815,"score_linear":89624.0,"score_lmetric":388485},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1789,"cache_hit":1024,"new_prefill":57791,"score_linear":54106.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1093,"cache_hit":0,"new_prefill":58815,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":993,"cache_hit":0,"new_prefill":58815,"score_linear":7065.0,"score_lmetric":58815},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":95317,"ongoing_decode_tokens":95317,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":973,"cache_hit":0,"new_prefill":58815,"score_linear":95317.0,"score_lmetric":117630}],"chosen_score_linear":104824.0,"chosen_score_lmetric":17278,"t_first_token":348691.806171905,"t_first_token_unix":1779779556.8201156,"t_done":348692.122405377,"t_done_unix":1779779557.1363487},{"request_id":"1341948:2:1347296:475","session_id":"1341948","input_length":25661,"t_proxy_recv":348690.798981146,"t_decision_unix":1779779555.8129227,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":19456,"affinity_cache_ratio":0.7581933673668212,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":19456,"estimated_new_tokens":6205,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":213815,"ongoing_decode_tokens":155000,"pending_prefill_tokens":8639,"num_requests":3,"active_p_offloads":0,"cached_blocks":992,"cache_hit":0,"new_prefill":25661,"score_linear":213815.0,"score_lmetric":102900},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":99637,"ongoing_decode_tokens":99637,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":963,"cache_hit":0,"new_prefill":25661,"score_linear":99637.0,"score_lmetric":51322},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1350,"cache_hit":0,"new_prefill":25661,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":89624,"ongoing_decode_tokens":75521,"pending_prefill_tokens":4887,"num_requests":3,"active_p_offloads":0,"cached_blocks":1088,"cache_hit":0,"new_prefill":25661,"score_linear":89624.0,"score_lmetric":91644},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1789,"cache_hit":0,"new_prefill":25661,"score_linear":55130.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1093,"cache_hit":0,"new_prefill":25661,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":993,"cache_hit":0,"new_prefill":25661,"score_linear":7065.0,"score_lmetric":25661},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":975,"cache_hit":19456,"new_prefill":6205,"score_linear":1915.0,"score_lmetric":6205}],"chosen_score_linear":1915.0,"chosen_score_lmetric":6205,"t_first_token":348691.83839713,"t_first_token_unix":1779779556.8523402,"t_done":348692.255285015,"t_done_unix":1779779557.2692285},{"request_id":"1341912:2:1343220:458","session_id":"1341912","input_length":10232,"t_proxy_recv":348676.840974043,"t_decision_unix":1779779541.8549156,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":9728,"affinity_cache_ratio":0.9507427677873339,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":9728,"estimated_new_tokens":504,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":59219,"ongoing_decode_tokens":0,"pending_prefill_tokens":59219,"num_requests":1,"active_p_offloads":0,"cached_blocks":992,"cache_hit":0,"new_prefill":10232,"score_linear":59219.0,"score_lmetric":69451},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":0,"new_prefill":10232,"score_linear":62618.0,"score_lmetric":10232},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":37527,"ongoing_decode_tokens":37527,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1281,"cache_hit":0,"new_prefill":10232,"score_linear":37527.0,"score_lmetric":10232},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":65289,"ongoing_decode_tokens":0,"pending_prefill_tokens":65289,"num_requests":1,"active_p_offloads":0,"cached_blocks":1088,"cache_hit":9728,"new_prefill":504,"score_linear":55561.0,"score_lmetric":65793},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":67042,"ongoing_decode_tokens":55130,"pending_prefill_tokens":11912,"num_requests":1,"active_p_offloads":0,"cached_blocks":1737,"cache_hit":0,"new_prefill":10232,"score_linear":67042.0,"score_lmetric":22144},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":66152,"ongoing_decode_tokens":66152,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1019,"cache_hit":0,"new_prefill":10232,"score_linear":66152.0,"score_lmetric":10232},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":993,"cache_hit":0,"new_prefill":10232,"score_linear":7065.0,"score_lmetric":10232},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":973,"cache_hit":0,"new_prefill":10232,"score_linear":21371.0,"score_lmetric":10232}],"chosen_score_linear":55561.0,"chosen_score_lmetric":65793,"t_first_token":348690.771485237,"t_first_token_unix":1779779555.7854285,"t_done":348692.725875878,"t_done_unix":1779779557.739819},{"request_id":"1342634:1:1342634:454","session_id":"1342634","input_length":59219,"t_proxy_recv":348674.726560787,"t_decision_unix":1779779539.7405026,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":59219,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":992,"cache_hit":0,"new_prefill":59219,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":0,"new_prefill":59219,"score_linear":62618.0,"score_lmetric":59219},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":58907,"ongoing_decode_tokens":0,"pending_prefill_tokens":14363,"num_requests":2,"active_p_offloads":0,"cached_blocks":1281,"cache_hit":0,"new_prefill":59219,"score_linear":58907.0,"score_lmetric":147164},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":2129,"ongoing_decode_tokens":2129,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1088,"cache_hit":0,"new_prefill":59219,"score_linear":2129.0,"score_lmetric":59219},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":103511,"ongoing_decode_tokens":55130,"pending_prefill_tokens":48381,"num_requests":1,"active_p_offloads":0,"cached_blocks":1643,"cache_hit":0,"new_prefill":59219,"score_linear":103511.0,"score_lmetric":107600},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":66152,"ongoing_decode_tokens":66152,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1019,"cache_hit":0,"new_prefill":59219,"score_linear":66152.0,"score_lmetric":59219},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":993,"cache_hit":0,"new_prefill":59219,"score_linear":7065.0,"score_lmetric":59219},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":41258,"ongoing_decode_tokens":41258,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":927,"cache_hit":0,"new_prefill":59219,"score_linear":41258.0,"score_lmetric":118438}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348687.607545227,"t_first_token_unix":1779779552.6214943,"t_done":348692.831249719,"t_done_unix":1779779557.8451931},{"request_id":"1347763:1:1347763:476","session_id":"1347763","input_length":5465,"t_proxy_recv":348692.406035576,"t_decision_unix":1779779557.419977,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":5465,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":155000,"ongoing_decode_tokens":155000,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1008,"cache_hit":0,"new_prefill":5465,"score_linear":155000.0,"score_lmetric":10930},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":968,"cache_hit":0,"new_prefill":5465,"score_linear":62618.0,"score_lmetric":5465},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1350,"cache_hit":0,"new_prefill":5465,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":75521,"ongoing_decode_tokens":75521,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1097,"cache_hit":0,"new_prefill":5465,"score_linear":75521.0,"score_lmetric":10930},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1789,"cache_hit":0,"new_prefill":5465,"score_linear":55130.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1093,"cache_hit":0,"new_prefill":5465,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1008,"cache_hit":0,"new_prefill":5465,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":987,"cache_hit":0,"new_prefill":5465,"score_linear":21371.0,"score_lmetric":5465}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348692.764628713,"t_first_token_unix":1779779557.778572,"t_done":348692.891468707,"t_done_unix":1779779557.9054124},{"request_id":"1279412:22:1344210:462","session_id":"1279412","input_length":95781,"t_proxy_recv":348680.06302086,"t_decision_unix":1779779545.0769622,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":95744,"affinity_cache_ratio":0.9996137020912289,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":95744,"estimated_new_tokens":37,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":59219,"ongoing_decode_tokens":0,"pending_prefill_tokens":59219,"num_requests":1,"active_p_offloads":0,"cached_blocks":992,"cache_hit":95744,"new_prefill":37,"score_linear":-36525.0,"score_lmetric":59256},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":0,"new_prefill":95781,"score_linear":62618.0,"score_lmetric":95781},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1349,"cache_hit":0,"new_prefill":95781,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":89624,"ongoing_decode_tokens":0,"pending_prefill_tokens":70680,"num_requests":3,"active_p_offloads":0,"cached_blocks":1088,"cache_hit":0,"new_prefill":95781,"score_linear":89624.0,"score_lmetric":499383},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1779,"cache_hit":0,"new_prefill":95781,"score_linear":55130.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1029,"cache_hit":0,"new_prefill":95781,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":993,"cache_hit":12800,"new_prefill":82981,"score_linear":-5735.0,"score_lmetric":82981},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":973,"cache_hit":12800,"new_prefill":82981,"score_linear":8571.0,"score_lmetric":82981}],"chosen_score_linear":-36525.0,"chosen_score_lmetric":59256,"t_first_token":348687.615105405,"t_first_token_unix":1779779552.6290498,"t_done":348693.774917457,"t_done_unix":1779779558.788861},{"request_id":"1342921:1:1342921:455","session_id":"1342921","input_length":65289,"t_proxy_recv":348675.771403216,"t_decision_unix":1779779540.7853446,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":65289,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":59219,"ongoing_decode_tokens":0,"pending_prefill_tokens":59219,"num_requests":1,"active_p_offloads":0,"cached_blocks":992,"cache_hit":0,"new_prefill":65289,"score_linear":59219.0,"score_lmetric":124508},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":0,"new_prefill":65289,"score_linear":62618.0,"score_lmetric":65289},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":58907,"ongoing_decode_tokens":58907,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1281,"cache_hit":0,"new_prefill":65289,"score_linear":58907.0,"score_lmetric":130578},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1088,"cache_hit":0,"new_prefill":65289,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":103511,"ongoing_decode_tokens":103511,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1643,"cache_hit":0,"new_prefill":65289,"score_linear":103511.0,"score_lmetric":65289},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":66152,"ongoing_decode_tokens":66152,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1019,"cache_hit":0,"new_prefill":65289,"score_linear":66152.0,"score_lmetric":65289},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":7065,"ongoing_decode_tokens":7065,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":993,"cache_hit":0,"new_prefill":65289,"score_linear":7065.0,"score_lmetric":65289},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":965,"cache_hit":0,"new_prefill":65289,"score_linear":21371.0,"score_lmetric":65289}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348690.770232133,"t_first_token_unix":1779779555.7841756,"t_done":348694.787974871,"t_done_unix":1779779559.8019183},{"request_id":"1345921:3:1348784:483","session_id":"1345921","input_length":1397,"t_proxy_recv":348696.07884872,"t_decision_unix":1779779561.0927904,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":2,"chosen_idx":5,"affinity_cache_hit":512,"affinity_cache_ratio":0.36649964209019326,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":1397,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135662,"ongoing_decode_tokens":135662,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1123,"cache_hit":0,"new_prefill":1397,"score_linear":135662.0,"score_lmetric":2794},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":968,"cache_hit":0,"new_prefill":1397,"score_linear":62618.0,"score_lmetric":1397},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":41202,"ongoing_decode_tokens":41202,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1350,"cache_hit":512,"new_prefill":885,"score_linear":40690.0,"score_lmetric":885},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":17217,"ongoing_decode_tokens":0,"pending_prefill_tokens":17217,"num_requests":1,"active_p_offloads":0,"cached_blocks":1226,"cache_hit":0,"new_prefill":1397,"score_linear":17217.0,"score_lmetric":18614},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":67831,"ongoing_decode_tokens":55130,"pending_prefill_tokens":12701,"num_requests":1,"active_p_offloads":0,"cached_blocks":1789,"cache_hit":0,"new_prefill":1397,"score_linear":67831.0,"score_lmetric":14098},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1093,"cache_hit":0,"new_prefill":1397,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1018,"cache_hit":0,"new_prefill":1397,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":987,"cache_hit":0,"new_prefill":1397,"score_linear":21371.0,"score_lmetric":1397}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348696.213154754,"t_first_token_unix":1779779561.2270982,"t_done":348696.332098048,"t_done_unix":1779779561.3460412},{"request_id":"1311753:3:1348389:480","session_id":"1311753","input_length":39636,"t_proxy_recv":348694.888615272,"t_decision_unix":1779779559.9025564,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":36352,"affinity_cache_ratio":0.9171460288626501,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":36352,"estimated_new_tokens":3284,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":96026,"ongoing_decode_tokens":96026,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1123,"cache_hit":36352,"new_prefill":3284,"score_linear":59674.0,"score_lmetric":3284},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":968,"cache_hit":0,"new_prefill":39636,"score_linear":62618.0,"score_lmetric":39636},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":41202,"ongoing_decode_tokens":0,"pending_prefill_tokens":3826,"num_requests":1,"active_p_offloads":0,"cached_blocks":1350,"cache_hit":0,"new_prefill":39636,"score_linear":41202.0,"score_lmetric":43462},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1226,"cache_hit":0,"new_prefill":39636,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1789,"cache_hit":19968,"new_prefill":19668,"score_linear":35162.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1093,"cache_hit":0,"new_prefill":39636,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1018,"cache_hit":0,"new_prefill":39636,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":987,"cache_hit":0,"new_prefill":39636,"score_linear":21371.0,"score_lmetric":39636}],"chosen_score_linear":59674.0,"chosen_score_lmetric":3284,"t_first_token":348695.809547411,"t_first_token_unix":1779779560.8234909,"t_done":348696.347901297,"t_done_unix":1779779561.3618448},{"request_id":"1348394:1:1348394:481","session_id":"1348394","input_length":17217,"t_proxy_recv":348694.915449336,"t_decision_unix":1779779559.929391,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":17217,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135662,"ongoing_decode_tokens":96026,"pending_prefill_tokens":3284,"num_requests":2,"active_p_offloads":0,"cached_blocks":1123,"cache_hit":0,"new_prefill":17217,"score_linear":135662.0,"score_lmetric":41002},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":968,"cache_hit":0,"new_prefill":17217,"score_linear":62618.0,"score_lmetric":17217},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":41202,"ongoing_decode_tokens":0,"pending_prefill_tokens":3826,"num_requests":1,"active_p_offloads":0,"cached_blocks":1350,"cache_hit":0,"new_prefill":17217,"score_linear":41202.0,"score_lmetric":21043},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1226,"cache_hit":0,"new_prefill":17217,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1789,"cache_hit":0,"new_prefill":17217,"score_linear":55130.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1093,"cache_hit":0,"new_prefill":17217,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1018,"cache_hit":0,"new_prefill":17217,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":987,"cache_hit":0,"new_prefill":17217,"score_linear":21371.0,"score_lmetric":17217}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348696.567058563,"t_first_token_unix":1779779561.581002,"t_done":348696.94785068,"t_done_unix":1779779561.961794},{"request_id":"1348691:1:1348691:482","session_id":"1348691","input_length":12701,"t_proxy_recv":348695.784656334,"t_decision_unix":1779779560.7985973,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":12701,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135662,"ongoing_decode_tokens":96026,"pending_prefill_tokens":3284,"num_requests":2,"active_p_offloads":0,"cached_blocks":1123,"cache_hit":0,"new_prefill":12701,"score_linear":135662.0,"score_lmetric":31970},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":968,"cache_hit":0,"new_prefill":12701,"score_linear":62618.0,"score_lmetric":12701},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":41202,"ongoing_decode_tokens":41202,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1350,"cache_hit":0,"new_prefill":12701,"score_linear":41202.0,"score_lmetric":12701},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":17217,"ongoing_decode_tokens":0,"pending_prefill_tokens":17217,"num_requests":1,"active_p_offloads":0,"cached_blocks":1226,"cache_hit":0,"new_prefill":12701,"score_linear":17217.0,"score_lmetric":29918},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1789,"cache_hit":0,"new_prefill":12701,"score_linear":55130.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1093,"cache_hit":0,"new_prefill":12701,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1018,"cache_hit":0,"new_prefill":12701,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":987,"cache_hit":0,"new_prefill":12701,"score_linear":21371.0,"score_lmetric":12701}],"chosen_score_linear":55130.0,"chosen_score_lmetric":0,"t_first_token":348696.857589496,"t_first_token_unix":1779779561.871533,"t_done":348697.503113432,"t_done_unix":1779779562.517057},{"request_id":"1344773:3:1349525:486","session_id":"1344773","input_length":33334,"t_proxy_recv":348698.713959082,"t_decision_unix":1779779563.7278998,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":32768,"affinity_cache_ratio":0.9830203395932081,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":32768,"estimated_new_tokens":566,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":96026,"ongoing_decode_tokens":96026,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1129,"cache_hit":0,"new_prefill":33334,"score_linear":96026.0,"score_lmetric":33334},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":968,"cache_hit":0,"new_prefill":33334,"score_linear":62618.0,"score_lmetric":33334},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":41202,"ongoing_decode_tokens":41202,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1350,"cache_hit":0,"new_prefill":33334,"score_linear":41202.0,"score_lmetric":33334},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1259,"cache_hit":0,"new_prefill":33334,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1813,"cache_hit":0,"new_prefill":33334,"score_linear":55130.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1095,"cache_hit":32768,"new_prefill":566,"score_linear":-32768.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1018,"cache_hit":0,"new_prefill":33334,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":97864,"ongoing_decode_tokens":21371,"pending_prefill_tokens":2253,"num_requests":2,"active_p_offloads":0,"cached_blocks":987,"cache_hit":0,"new_prefill":33334,"score_linear":97864.0,"score_lmetric":71174}],"chosen_score_linear":-32768.0,"chosen_score_lmetric":0,"t_first_token":348698.909764193,"t_first_token_unix":1779779563.923708,"t_done":348700.357206454,"t_done_unix":1779779565.3711495},{"request_id":"1349975:1:1349975:487","session_id":"1349975","input_length":4988,"t_proxy_recv":348700.302621673,"t_decision_unix":1779779565.3165624,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":4988,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":96026,"ongoing_decode_tokens":96026,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1129,"cache_hit":0,"new_prefill":4988,"score_linear":96026.0,"score_lmetric":4988},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":968,"cache_hit":0,"new_prefill":4988,"score_linear":62618.0,"score_lmetric":4988},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":41202,"ongoing_decode_tokens":41202,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1350,"cache_hit":0,"new_prefill":4988,"score_linear":41202.0,"score_lmetric":4988},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1259,"cache_hit":0,"new_prefill":4988,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1813,"cache_hit":0,"new_prefill":4988,"score_linear":55130.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":33334,"ongoing_decode_tokens":33334,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1095,"cache_hit":0,"new_prefill":4988,"score_linear":33334.0,"score_lmetric":4988},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1018,"cache_hit":0,"new_prefill":4988,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":97864,"ongoing_decode_tokens":97864,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":987,"cache_hit":0,"new_prefill":4988,"score_linear":97864.0,"score_lmetric":9976}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348700.635092352,"t_first_token_unix":1779779565.649036,"t_done":348700.956201074,"t_done_unix":1779779565.9701443},{"request_id":"1350162:1:1350162:488","session_id":"1350162","input_length":4594,"t_proxy_recv":348700.987599049,"t_decision_unix":1779779566.0015404,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":4594,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":96026,"ongoing_decode_tokens":96026,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1129,"cache_hit":0,"new_prefill":4594,"score_linear":96026.0,"score_lmetric":4594},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":968,"cache_hit":0,"new_prefill":4594,"score_linear":62618.0,"score_lmetric":4594},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":41202,"ongoing_decode_tokens":41202,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1350,"cache_hit":0,"new_prefill":4594,"score_linear":41202.0,"score_lmetric":4594},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1259,"cache_hit":0,"new_prefill":4594,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1813,"cache_hit":0,"new_prefill":4594,"score_linear":55130.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1096,"cache_hit":0,"new_prefill":4594,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1027,"cache_hit":0,"new_prefill":4594,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":97864,"ongoing_decode_tokens":97864,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":987,"cache_hit":0,"new_prefill":4594,"score_linear":97864.0,"score_lmetric":9188}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348701.274562033,"t_first_token_unix":1779779566.2885053,"t_done":348701.390809858,"t_done_unix":1779779566.4047537},{"request_id":"1334223:3:1348241:479","session_id":"1334223","input_length":41202,"t_proxy_recv":348694.319024404,"t_decision_unix":1779779559.3329656,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":37376,"affinity_cache_ratio":0.9071404300762099,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":37376,"estimated_new_tokens":3826,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":96026,"ongoing_decode_tokens":96026,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1123,"cache_hit":0,"new_prefill":41202,"score_linear":96026.0,"score_lmetric":41202},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":968,"cache_hit":0,"new_prefill":41202,"score_linear":62618.0,"score_lmetric":41202},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1350,"cache_hit":37376,"new_prefill":3826,"score_linear":-37376.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":65289,"ongoing_decode_tokens":65289,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1098,"cache_hit":0,"new_prefill":41202,"score_linear":65289.0,"score_lmetric":41202},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1789,"cache_hit":0,"new_prefill":41202,"score_linear":55130.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1093,"cache_hit":0,"new_prefill":41202,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1018,"cache_hit":0,"new_prefill":41202,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":987,"cache_hit":0,"new_prefill":41202,"score_linear":21371.0,"score_lmetric":41202}],"chosen_score_linear":-37376.0,"chosen_score_lmetric":0,"t_first_token":348695.293639281,"t_first_token_unix":1779779560.307583,"t_done":348703.419563265,"t_done_unix":1779779568.433507},{"request_id":"1269373:19:1349406:485","session_id":"1269373","input_length":76493,"t_proxy_recv":348698.36558067,"t_decision_unix":1779779563.3795218,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":74240,"affinity_cache_ratio":0.9705463245002811,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":74240,"estimated_new_tokens":2253,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":96026,"ongoing_decode_tokens":96026,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1129,"cache_hit":7680,"new_prefill":68813,"score_linear":88346.0,"score_lmetric":68813},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":968,"cache_hit":0,"new_prefill":76493,"score_linear":62618.0,"score_lmetric":76493},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":41202,"ongoing_decode_tokens":41202,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1350,"cache_hit":0,"new_prefill":76493,"score_linear":41202.0,"score_lmetric":76493},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1259,"cache_hit":0,"new_prefill":76493,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1813,"cache_hit":7680,"new_prefill":68813,"score_linear":47450.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1095,"cache_hit":0,"new_prefill":76493,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1018,"cache_hit":7680,"new_prefill":68813,"score_linear":-7680.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":987,"cache_hit":74240,"new_prefill":2253,"score_linear":-52869.0,"score_lmetric":2253}],"chosen_score_linear":-52869.0,"chosen_score_lmetric":2253,"t_first_token":348699.590245253,"t_first_token_unix":1779779564.6041896,"t_done":348703.867538233,"t_done_unix":1779779568.8814816},{"request_id":"1350896:1:1350896:489","session_id":"1350896","input_length":1471,"t_proxy_recv":348703.435181339,"t_decision_unix":1779779568.449123,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":1471,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":96026,"ongoing_decode_tokens":96026,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1129,"cache_hit":0,"new_prefill":1471,"score_linear":96026.0,"score_lmetric":1471},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":968,"cache_hit":0,"new_prefill":1471,"score_linear":62618.0,"score_lmetric":1471},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1359,"cache_hit":0,"new_prefill":1471,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1268,"cache_hit":0,"new_prefill":1471,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1813,"cache_hit":0,"new_prefill":1471,"score_linear":55130.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1096,"cache_hit":0,"new_prefill":1471,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1027,"cache_hit":0,"new_prefill":1471,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":97864,"ongoing_decode_tokens":97864,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":987,"cache_hit":0,"new_prefill":1471,"score_linear":97864.0,"score_lmetric":2942}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348703.534655818,"t_first_token_unix":1779779568.5485995,"t_done":348703.904989378,"t_done_unix":1779779568.9189327},{"request_id":"1350980:1:1350980:490","session_id":"1350980","input_length":15313,"t_proxy_recv":348703.683049275,"t_decision_unix":1779779568.696991,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":15313,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":96026,"ongoing_decode_tokens":96026,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1129,"cache_hit":0,"new_prefill":15313,"score_linear":96026.0,"score_lmetric":15313},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":968,"cache_hit":0,"new_prefill":15313,"score_linear":62618.0,"score_lmetric":15313},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":1471,"ongoing_decode_tokens":1471,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1359,"cache_hit":0,"new_prefill":15313,"score_linear":1471.0,"score_lmetric":15313},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1268,"cache_hit":0,"new_prefill":15313,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1813,"cache_hit":0,"new_prefill":15313,"score_linear":55130.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1096,"cache_hit":0,"new_prefill":15313,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1027,"cache_hit":0,"new_prefill":15313,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":97864,"ongoing_decode_tokens":97864,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":987,"cache_hit":0,"new_prefill":15313,"score_linear":97864.0,"score_lmetric":30626}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348705.056695953,"t_first_token_unix":1779779570.0706398,"t_done":348705.088638732,"t_done_unix":1779779570.102582},{"request_id":"1253743:5:1351907:492","session_id":"1253743","input_length":51340,"t_proxy_recv":348707.221054945,"t_decision_unix":1779779572.2349963,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":50176,"affinity_cache_ratio":0.9773276197896377,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":50176,"estimated_new_tokens":1164,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":96026,"ongoing_decode_tokens":96026,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1129,"cache_hit":0,"new_prefill":51340,"score_linear":96026.0,"score_lmetric":51340},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":968,"cache_hit":4608,"new_prefill":46732,"score_linear":58010.0,"score_lmetric":46732},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1362,"cache_hit":0,"new_prefill":51340,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1268,"cache_hit":50176,"new_prefill":1164,"score_linear":-50176.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1813,"cache_hit":0,"new_prefill":51340,"score_linear":55130.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1125,"cache_hit":17408,"new_prefill":33932,"score_linear":-17408.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1027,"cache_hit":0,"new_prefill":51340,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":992,"cache_hit":0,"new_prefill":51340,"score_linear":21371.0,"score_lmetric":51340}],"chosen_score_linear":-50176.0,"chosen_score_lmetric":0,"t_first_token":348707.609428802,"t_first_token_unix":1779779572.6233723,"t_done":348708.034335733,"t_done_unix":1779779573.0482793},{"request_id":"1344773:4:1352462:494","session_id":"1344773","input_length":33579,"t_proxy_recv":348709.017183056,"t_decision_unix":1779779574.031124,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":33280,"affinity_cache_ratio":0.9910956252419667,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":33280,"estimated_new_tokens":299,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":96026,"ongoing_decode_tokens":96026,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1129,"cache_hit":0,"new_prefill":33579,"score_linear":96026.0,"score_lmetric":33579},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":968,"cache_hit":0,"new_prefill":33579,"score_linear":62618.0,"score_lmetric":33579},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1362,"cache_hit":0,"new_prefill":33579,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1270,"cache_hit":0,"new_prefill":33579,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":76566,"ongoing_decode_tokens":55130,"pending_prefill_tokens":9148,"num_requests":1,"active_p_offloads":0,"cached_blocks":1813,"cache_hit":0,"new_prefill":33579,"score_linear":76566.0,"score_lmetric":42727},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1125,"cache_hit":33280,"new_prefill":299,"score_linear":-33280.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1027,"cache_hit":0,"new_prefill":33579,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":992,"cache_hit":0,"new_prefill":33579,"score_linear":21371.0,"score_lmetric":33579}],"chosen_score_linear":-33280.0,"chosen_score_lmetric":0,"t_first_token":348709.092989835,"t_first_token_unix":1779779574.1069329,"t_done":348709.257755024,"t_done_unix":1779779574.2716982},{"request_id":"1350162:2:1352468:495","session_id":"1350162","input_length":10575,"t_proxy_recv":348709.026768217,"t_decision_unix":1779779574.04071,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":4608,"affinity_cache_ratio":0.4357446808510638,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":4608,"estimated_new_tokens":5967,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":96026,"ongoing_decode_tokens":96026,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1129,"cache_hit":0,"new_prefill":10575,"score_linear":96026.0,"score_lmetric":10575},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":968,"cache_hit":0,"new_prefill":10575,"score_linear":62618.0,"score_lmetric":10575},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1362,"cache_hit":0,"new_prefill":10575,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1270,"cache_hit":4608,"new_prefill":5967,"score_linear":-4608.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":76566,"ongoing_decode_tokens":55130,"pending_prefill_tokens":9148,"num_requests":1,"active_p_offloads":0,"cached_blocks":1813,"cache_hit":0,"new_prefill":10575,"score_linear":76566.0,"score_lmetric":19723},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":33579,"ongoing_decode_tokens":0,"pending_prefill_tokens":299,"num_requests":1,"active_p_offloads":0,"cached_blocks":1125,"cache_hit":0,"new_prefill":10575,"score_linear":33579.0,"score_lmetric":10874},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1027,"cache_hit":0,"new_prefill":10575,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":992,"cache_hit":0,"new_prefill":10575,"score_linear":21371.0,"score_lmetric":10575}],"chosen_score_linear":-4608.0,"chosen_score_lmetric":0,"t_first_token":348709.58012381,"t_first_token_unix":1779779574.594067,"t_done":348709.705027027,"t_done_unix":1779779574.7189698},{"request_id":"1348691:2:1352277:493","session_id":"1348691","input_length":21436,"t_proxy_recv":348708.402885761,"t_decision_unix":1779779573.4168272,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":12288,"affinity_cache_ratio":0.5732412763575294,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":12288,"estimated_new_tokens":9148,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":96026,"ongoing_decode_tokens":96026,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1129,"cache_hit":0,"new_prefill":21436,"score_linear":96026.0,"score_lmetric":21436},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":968,"cache_hit":0,"new_prefill":21436,"score_linear":62618.0,"score_lmetric":21436},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1362,"cache_hit":0,"new_prefill":21436,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1270,"cache_hit":0,"new_prefill":21436,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1813,"cache_hit":12288,"new_prefill":9148,"score_linear":42842.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1125,"cache_hit":0,"new_prefill":21436,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1027,"cache_hit":0,"new_prefill":21436,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":992,"cache_hit":0,"new_prefill":21436,"score_linear":21371.0,"score_lmetric":21436}],"chosen_score_linear":42842.0,"chosen_score_lmetric":0,"t_first_token":348709.701877587,"t_first_token_unix":1779779574.7158234,"t_done":348710.093998321,"t_done_unix":1779779575.1079533},{"request_id":"1345921:4:1353010:501","session_id":"1345921","input_length":6935,"t_proxy_recv":348711.006225337,"t_decision_unix":1779779576.0201666,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":5,"chosen_idx":3,"affinity_cache_hit":1024,"affinity_cache_ratio":0.14765681326604183,"affinity_num_requests":2,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":6935,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":96026,"ongoing_decode_tokens":96026,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1129,"cache_hit":0,"new_prefill":6935,"score_linear":96026.0,"score_lmetric":6935},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":968,"cache_hit":0,"new_prefill":6935,"score_linear":62618.0,"score_lmetric":6935},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":18840,"ongoing_decode_tokens":0,"pending_prefill_tokens":18840,"num_requests":1,"active_p_offloads":0,"cached_blocks":1362,"cache_hit":512,"new_prefill":6423,"score_linear":18328.0,"score_lmetric":25263},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1281,"cache_hit":0,"new_prefill":6935,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1830,"cache_hit":0,"new_prefill":6935,"score_linear":55130.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":151585,"ongoing_decode_tokens":68259,"pending_prefill_tokens":382,"num_requests":2,"active_p_offloads":0,"cached_blocks":1125,"cache_hit":1024,"new_prefill":5911,"score_linear":150561.0,"score_lmetric":12586},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":14239,"ongoing_decode_tokens":0,"pending_prefill_tokens":9631,"num_requests":1,"active_p_offloads":0,"cached_blocks":1027,"cache_hit":0,"new_prefill":6935,"score_linear":14239.0,"score_lmetric":16566},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":99161,"ongoing_decode_tokens":99161,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":992,"cache_hit":0,"new_prefill":6935,"score_linear":99161.0,"score_lmetric":13870}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348711.467578675,"t_first_token_unix":1779779576.4815218,"t_done":348711.469720666,"t_done_unix":1779779576.4836636},{"request_id":"1352606:1:1352606:496","session_id":"1352606","input_length":18840,"t_proxy_recv":348709.594635261,"t_decision_unix":1779779574.6085768,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":18840,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":96026,"ongoing_decode_tokens":96026,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1129,"cache_hit":0,"new_prefill":18840,"score_linear":96026.0,"score_lmetric":18840},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":968,"cache_hit":0,"new_prefill":18840,"score_linear":62618.0,"score_lmetric":18840},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1362,"cache_hit":0,"new_prefill":18840,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":10575,"ongoing_decode_tokens":10575,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1270,"cache_hit":0,"new_prefill":18840,"score_linear":10575.0,"score_lmetric":18840},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":76566,"ongoing_decode_tokens":55130,"pending_prefill_tokens":9148,"num_requests":1,"active_p_offloads":0,"cached_blocks":1813,"cache_hit":0,"new_prefill":18840,"score_linear":76566.0,"score_lmetric":27988},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1125,"cache_hit":0,"new_prefill":18840,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1027,"cache_hit":0,"new_prefill":18840,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":992,"cache_hit":0,"new_prefill":18840,"score_linear":21371.0,"score_lmetric":18840}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348711.471141087,"t_first_token_unix":1779779576.485084,"t_done":348711.921645076,"t_done_unix":1779779576.9355884},{"request_id":"1349975:2:1352986:500","session_id":"1349975","input_length":14239,"t_proxy_recv":348710.971556323,"t_decision_unix":1779779575.9854977,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":4608,"affinity_cache_ratio":0.32361823161738884,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":4608,"estimated_new_tokens":9631,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":96026,"ongoing_decode_tokens":96026,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1129,"cache_hit":0,"new_prefill":14239,"score_linear":96026.0,"score_lmetric":14239},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":968,"cache_hit":0,"new_prefill":14239,"score_linear":62618.0,"score_lmetric":14239},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":18840,"ongoing_decode_tokens":0,"pending_prefill_tokens":18840,"num_requests":1,"active_p_offloads":0,"cached_blocks":1362,"cache_hit":0,"new_prefill":14239,"score_linear":18840.0,"score_lmetric":33079},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1281,"cache_hit":0,"new_prefill":14239,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1830,"cache_hit":0,"new_prefill":14239,"score_linear":55130.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":151585,"ongoing_decode_tokens":68259,"pending_prefill_tokens":382,"num_requests":2,"active_p_offloads":0,"cached_blocks":1125,"cache_hit":0,"new_prefill":14239,"score_linear":151585.0,"score_lmetric":29242},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1027,"cache_hit":4608,"new_prefill":9631,"score_linear":-4608.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":99161,"ongoing_decode_tokens":99161,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":992,"cache_hit":0,"new_prefill":14239,"score_linear":99161.0,"score_lmetric":28478}],"chosen_score_linear":-4608.0,"chosen_score_lmetric":0,"t_first_token":348711.993123527,"t_first_token_unix":1779779577.0070672,"t_done":348712.185329561,"t_done_unix":1779779577.1992726},{"request_id":"1317598:6:1352957:499","session_id":"1317598","input_length":83326,"t_proxy_recv":348710.871337095,"t_decision_unix":1779779575.885278,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":82944,"affinity_cache_ratio":0.9954155965724983,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":82944,"estimated_new_tokens":382,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":96026,"ongoing_decode_tokens":96026,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1129,"cache_hit":0,"new_prefill":83326,"score_linear":96026.0,"score_lmetric":83326},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":968,"cache_hit":0,"new_prefill":83326,"score_linear":62618.0,"score_lmetric":83326},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":18840,"ongoing_decode_tokens":0,"pending_prefill_tokens":18840,"num_requests":1,"active_p_offloads":0,"cached_blocks":1362,"cache_hit":0,"new_prefill":83326,"score_linear":18840.0,"score_lmetric":102166},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1281,"cache_hit":0,"new_prefill":83326,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1830,"cache_hit":512,"new_prefill":82814,"score_linear":54618.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1125,"cache_hit":82944,"new_prefill":382,"score_linear":-14685.0,"score_lmetric":382},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1027,"cache_hit":0,"new_prefill":83326,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":99161,"ongoing_decode_tokens":21371,"pending_prefill_tokens":990,"num_requests":2,"active_p_offloads":0,"cached_blocks":992,"cache_hit":0,"new_prefill":83326,"score_linear":99161.0,"score_lmetric":168632}],"chosen_score_linear":-14685.0,"chosen_score_lmetric":382,"t_first_token":348711.172293605,"t_first_token_unix":1779779576.1862369,"t_done":348712.872242517,"t_done_unix":1779779577.886186},{"request_id":"1315567:3:1325346:372","session_id":"1315567","input_length":55130,"t_proxy_recv":348613.571731502,"t_decision_unix":1779779478.585672,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":54784,"affinity_cache_ratio":0.9937239252675494,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":54784,"estimated_new_tokens":346,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":26994,"ongoing_decode_tokens":26994,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":807,"cache_hit":7680,"new_prefill":47450,"score_linear":19314.0,"score_lmetric":47450},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":948,"cache_hit":0,"new_prefill":55130,"score_linear":62618.0,"score_lmetric":55130},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":966,"cache_hit":0,"new_prefill":55130,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":932,"cache_hit":0,"new_prefill":55130,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":4312,"ongoing_decode_tokens":4312,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1556,"cache_hit":54784,"new_prefill":346,"score_linear":-50472.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":79663,"ongoing_decode_tokens":79663,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":661,"cache_hit":0,"new_prefill":55130,"score_linear":79663.0,"score_lmetric":55130},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":33182,"ongoing_decode_tokens":33182,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":845,"cache_hit":17408,"new_prefill":37722,"score_linear":15774.0,"score_lmetric":37722},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":188103,"ongoing_decode_tokens":188103,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":834,"cache_hit":7680,"new_prefill":47450,"score_linear":180423.0,"score_lmetric":94900}],"chosen_score_linear":-50472.0,"chosen_score_lmetric":0,"t_first_token":348613.760306963,"t_first_token_unix":1779779478.7742505,"t_done":348713.357299124,"t_done_unix":1779779578.3712425},{"request_id":"1269373:20:1352797:498","session_id":"1269373","input_length":77790,"t_proxy_recv":348710.270132602,"t_decision_unix":1779779575.2840738,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":76800,"affinity_cache_ratio":0.9872734284612418,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":76800,"estimated_new_tokens":990,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":96026,"ongoing_decode_tokens":96026,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1129,"cache_hit":7680,"new_prefill":70110,"score_linear":88346.0,"score_lmetric":70110},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":968,"cache_hit":0,"new_prefill":77790,"score_linear":62618.0,"score_lmetric":77790},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":18840,"ongoing_decode_tokens":0,"pending_prefill_tokens":18840,"num_requests":1,"active_p_offloads":0,"cached_blocks":1362,"cache_hit":0,"new_prefill":77790,"score_linear":18840.0,"score_lmetric":96630},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1281,"cache_hit":0,"new_prefill":77790,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1830,"cache_hit":7680,"new_prefill":70110,"score_linear":47450.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1125,"cache_hit":0,"new_prefill":77790,"score_linear":68259.0,"score_lmetric":77790},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1027,"cache_hit":7680,"new_prefill":70110,"score_linear":-7680.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":992,"cache_hit":76800,"new_prefill":990,"score_linear":-55429.0,"score_lmetric":990}],"chosen_score_linear":-55429.0,"chosen_score_lmetric":990,"t_first_token":348710.87877361,"t_first_token_unix":1779779575.8927166,"t_done":348713.583277788,"t_done_unix":1779779578.597221},{"request_id":"1279412:23:1348089:477","session_id":"1279412","input_length":96026,"t_proxy_recv":348693.791346506,"t_decision_unix":1779779558.805288,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":95744,"affinity_cache_ratio":0.9970632953575074,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":95744,"estimated_new_tokens":282,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1123,"cache_hit":95744,"new_prefill":282,"score_linear":-95744.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":968,"cache_hit":0,"new_prefill":96026,"score_linear":62618.0,"score_lmetric":96026},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1350,"cache_hit":0,"new_prefill":96026,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":65289,"ongoing_decode_tokens":65289,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1098,"cache_hit":0,"new_prefill":96026,"score_linear":65289.0,"score_lmetric":96026},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1789,"cache_hit":0,"new_prefill":96026,"score_linear":55130.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1093,"cache_hit":0,"new_prefill":96026,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1018,"cache_hit":12800,"new_prefill":83226,"score_linear":-12800.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":987,"cache_hit":12800,"new_prefill":83226,"score_linear":8571.0,"score_lmetric":83226}],"chosen_score_linear":-95744.0,"chosen_score_lmetric":0,"t_first_token":348693.939599631,"t_first_token_unix":1779779558.9535437,"t_done":348715.206255891,"t_done_unix":1779779580.2201996},{"request_id":"1344773:5:1353365:502","session_id":"1344773","input_length":33875,"t_proxy_recv":348712.484636252,"t_decision_unix":1779779577.4985774,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":33280,"affinity_cache_ratio":0.9824354243542436,"affinity_num_requests":2,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":33280,"estimated_new_tokens":595,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":96026,"ongoing_decode_tokens":96026,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1129,"cache_hit":0,"new_prefill":33875,"score_linear":96026.0,"score_lmetric":33875},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":968,"cache_hit":0,"new_prefill":33875,"score_linear":62618.0,"score_lmetric":33875},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1398,"cache_hit":0,"new_prefill":33875,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1294,"cache_hit":0,"new_prefill":33875,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1830,"cache_hit":0,"new_prefill":33875,"score_linear":55130.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":151585,"ongoing_decode_tokens":151585,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1125,"cache_hit":33280,"new_prefill":595,"score_linear":118305.0,"score_lmetric":1190},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1045,"cache_hit":0,"new_prefill":33875,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":99161,"ongoing_decode_tokens":99161,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":992,"cache_hit":0,"new_prefill":33875,"score_linear":99161.0,"score_lmetric":67750}],"chosen_score_linear":118305.0,"chosen_score_lmetric":1190,"t_first_token":348712.68796233,"t_first_token_unix":1779779577.701906,"t_done":348715.774208622,"t_done_unix":1779779580.7881522},{"request_id":"1353821:1:1353821:503","session_id":"1353821","input_length":27651,"t_proxy_recv":348713.889685171,"t_decision_unix":1779779578.9036262,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":-27651,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":27651,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":96026,"ongoing_decode_tokens":96026,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1129,"cache_hit":0,"new_prefill":27651,"score_linear":96026.0,"score_lmetric":27651},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":968,"cache_hit":0,"new_prefill":27651,"score_linear":62618.0,"score_lmetric":27651},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1398,"cache_hit":0,"new_prefill":27651,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1294,"cache_hit":0,"new_prefill":27651,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1832,"cache_hit":0,"new_prefill":27651,"score_linear":0.0,"score_lmetric":-27651},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":102134,"ongoing_decode_tokens":102134,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1125,"cache_hit":0,"new_prefill":27651,"score_linear":102134.0,"score_lmetric":55302},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1045,"cache_hit":0,"new_prefill":27651,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":994,"cache_hit":0,"new_prefill":27651,"score_linear":21371.0,"score_lmetric":27651}],"chosen_score_linear":0.0,"chosen_score_lmetric":-27651,"t_first_token":348717.331953967,"t_first_token_unix":1779779582.3458977,"t_done":348718.148955618,"t_done_unix":1779779583.1628993},{"request_id":"1355484:1:1355484:508","session_id":"1355484","input_length":19366,"t_proxy_recv":348719.597248274,"t_decision_unix":1779779584.6111894,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":12800,"estimated_new_tokens":6566,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1131,"cache_hit":12800,"new_prefill":6566,"score_linear":-12800.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":968,"cache_hit":0,"new_prefill":19366,"score_linear":62618.0,"score_lmetric":19366},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1398,"cache_hit":0,"new_prefill":19366,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":134962,"ongoing_decode_tokens":0,"pending_prefill_tokens":83762,"num_requests":2,"active_p_offloads":0,"cached_blocks":1294,"cache_hit":0,"new_prefill":19366,"score_linear":134962.0,"score_lmetric":206256},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":24503,"ongoing_decode_tokens":0,"pending_prefill_tokens":24503,"num_requests":0,"active_p_offloads":0,"cached_blocks":1882,"cache_hit":0,"new_prefill":19366,"score_linear":24503.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":83778,"ongoing_decode_tokens":83778,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1126,"cache_hit":0,"new_prefill":19366,"score_linear":83778.0,"score_lmetric":38732},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1045,"cache_hit":12800,"new_prefill":6566,"score_linear":-12800.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":994,"cache_hit":12800,"new_prefill":6566,"score_linear":8571.0,"score_lmetric":6566}],"chosen_score_linear":-12800.0,"chosen_score_lmetric":0,"t_first_token":348720.535449393,"t_first_token_unix":1779779585.5493927,"t_done":348721.138198641,"t_done_unix":1779779586.1521423},{"request_id":"1286804:3:1354910:506","session_id":"1286804","input_length":15519,"t_proxy_recv":348717.692215892,"t_decision_unix":1779779582.706157,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":13824,"affinity_cache_ratio":0.8907790450415619,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":13824,"estimated_new_tokens":1695,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1131,"cache_hit":0,"new_prefill":15519,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":968,"cache_hit":0,"new_prefill":15519,"score_linear":62618.0,"score_lmetric":15519},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1398,"cache_hit":0,"new_prefill":15519,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":134962,"ongoing_decode_tokens":0,"pending_prefill_tokens":83762,"num_requests":2,"active_p_offloads":0,"cached_blocks":1294,"cache_hit":0,"new_prefill":15519,"score_linear":134962.0,"score_lmetric":198562},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27651,"ongoing_decode_tokens":27651,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1832,"cache_hit":0,"new_prefill":15519,"score_linear":27651.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1126,"cache_hit":13824,"new_prefill":1695,"score_linear":54435.0,"score_lmetric":1695},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1045,"cache_hit":0,"new_prefill":15519,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":994,"cache_hit":0,"new_prefill":15519,"score_linear":21371.0,"score_lmetric":15519}],"chosen_score_linear":54435.0,"chosen_score_lmetric":1695,"t_first_token":348719.09821184,"t_first_token_unix":1779779584.112156,"t_done":348722.00154464,"t_done_unix":1779779587.0154877},{"request_id":"1344773:6:1356097:513","session_id":"1344773","input_length":34198,"t_proxy_recv":348721.852881561,"t_decision_unix":1779779586.866823,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":33792,"affinity_cache_ratio":0.9881279606994561,"affinity_num_requests":2,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":33792,"estimated_new_tokens":406,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":34198,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":968,"cache_hit":0,"new_prefill":34198,"score_linear":62618.0,"score_lmetric":34198},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":35308,"ongoing_decode_tokens":0,"pending_prefill_tokens":35308,"num_requests":1,"active_p_offloads":0,"cached_blocks":1398,"cache_hit":0,"new_prefill":34198,"score_linear":35308.0,"score_lmetric":69506},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":134962,"ongoing_decode_tokens":0,"pending_prefill_tokens":83762,"num_requests":2,"active_p_offloads":0,"cached_blocks":1294,"cache_hit":0,"new_prefill":34198,"score_linear":134962.0,"score_lmetric":235920},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121673,"ongoing_decode_tokens":0,"pending_prefill_tokens":28489,"num_requests":2,"active_p_offloads":0,"cached_blocks":1882,"cache_hit":0,"new_prefill":34198,"score_linear":121673.0,"score_lmetric":125374},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":83778,"ongoing_decode_tokens":83778,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1126,"cache_hit":33792,"new_prefill":406,"score_linear":49986.0,"score_lmetric":812},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1045,"cache_hit":0,"new_prefill":34198,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":99460,"ongoing_decode_tokens":99460,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":994,"cache_hit":0,"new_prefill":34198,"score_linear":99460.0,"score_lmetric":68396}],"chosen_score_linear":49986.0,"chosen_score_lmetric":812,"t_first_token":348721.974449101,"t_first_token_unix":1779779586.988392,"t_done":348722.249830744,"t_done_unix":1779779587.263774},{"request_id":"1269373:21:1355702:509","session_id":"1269373","input_length":78089,"t_proxy_recv":348720.280264032,"t_decision_unix":1779779585.294205,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":77824,"affinity_cache_ratio":0.9966064362458221,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":77824,"estimated_new_tokens":265,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":19366,"ongoing_decode_tokens":0,"pending_prefill_tokens":6566,"num_requests":1,"active_p_offloads":0,"cached_blocks":1131,"cache_hit":7680,"new_prefill":70409,"score_linear":11686.0,"score_lmetric":76975},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":968,"cache_hit":0,"new_prefill":78089,"score_linear":62618.0,"score_lmetric":78089},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1398,"cache_hit":0,"new_prefill":78089,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":134962,"ongoing_decode_tokens":0,"pending_prefill_tokens":83762,"num_requests":2,"active_p_offloads":0,"cached_blocks":1294,"cache_hit":0,"new_prefill":78089,"score_linear":134962.0,"score_lmetric":323702},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":24503,"ongoing_decode_tokens":0,"pending_prefill_tokens":24503,"num_requests":0,"active_p_offloads":0,"cached_blocks":1882,"cache_hit":7680,"new_prefill":70409,"score_linear":16823.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":83778,"ongoing_decode_tokens":83778,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1126,"cache_hit":0,"new_prefill":78089,"score_linear":83778.0,"score_lmetric":156178},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1045,"cache_hit":7680,"new_prefill":70409,"score_linear":-7680.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":994,"cache_hit":77824,"new_prefill":265,"score_linear":-56453.0,"score_lmetric":265}],"chosen_score_linear":-56453.0,"chosen_score_lmetric":265,"t_first_token":348720.452773447,"t_first_token_unix":1779779585.4667168,"t_done":348725.234321629,"t_done_unix":1779779590.2482653},{"request_id":"1356948:1:1356948:515","session_id":"1356948","input_length":6711,"t_proxy_recv":348724.880833766,"t_decision_unix":1779779589.8947744,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":6711,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":122597,"ongoing_decode_tokens":0,"pending_prefill_tokens":121573,"num_requests":1,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":6711,"score_linear":122597.0,"score_lmetric":128284},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":968,"cache_hit":0,"new_prefill":6711,"score_linear":62618.0,"score_lmetric":6711},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":35308,"ongoing_decode_tokens":0,"pending_prefill_tokens":35308,"num_requests":1,"active_p_offloads":0,"cached_blocks":1398,"cache_hit":0,"new_prefill":6711,"score_linear":35308.0,"score_lmetric":42019},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":134962,"ongoing_decode_tokens":0,"pending_prefill_tokens":83762,"num_requests":2,"active_p_offloads":0,"cached_blocks":1294,"cache_hit":0,"new_prefill":6711,"score_linear":134962.0,"score_lmetric":180946},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121673,"ongoing_decode_tokens":24503,"pending_prefill_tokens":3986,"num_requests":2,"active_p_offloads":0,"cached_blocks":1882,"cache_hit":0,"new_prefill":6711,"score_linear":121673.0,"score_lmetric":21394},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1129,"cache_hit":0,"new_prefill":6711,"score_linear":68259.0,"score_lmetric":6711},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1045,"cache_hit":0,"new_prefill":6711,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":99460,"ongoing_decode_tokens":99460,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":994,"cache_hit":0,"new_prefill":6711,"score_linear":99460.0,"score_lmetric":13422}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348725.349660074,"t_first_token_unix":1779779590.3636038,"t_done":348725.621178863,"t_done_unix":1779779590.635122},{"request_id":"1355951:1:1355951:511","session_id":"1355951","input_length":35308,"t_proxy_recv":348721.190476316,"t_decision_unix":1779779586.2044177,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":35308,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":35308,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":968,"cache_hit":0,"new_prefill":35308,"score_linear":62618.0,"score_lmetric":35308},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1398,"cache_hit":0,"new_prefill":35308,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":134962,"ongoing_decode_tokens":0,"pending_prefill_tokens":83762,"num_requests":2,"active_p_offloads":0,"cached_blocks":1294,"cache_hit":0,"new_prefill":35308,"score_linear":134962.0,"score_lmetric":238140},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":72696,"ongoing_decode_tokens":0,"pending_prefill_tokens":27640,"num_requests":1,"active_p_offloads":0,"cached_blocks":1882,"cache_hit":0,"new_prefill":35308,"score_linear":72696.0,"score_lmetric":62948},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":83778,"ongoing_decode_tokens":83778,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1126,"cache_hit":0,"new_prefill":35308,"score_linear":83778.0,"score_lmetric":70616},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1045,"cache_hit":0,"new_prefill":35308,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":99460,"ongoing_decode_tokens":99460,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":994,"cache_hit":0,"new_prefill":35308,"score_linear":99460.0,"score_lmetric":70616}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348726.359475142,"t_first_token_unix":1779779591.373419,"t_done":348726.756741489,"t_done_unix":1779779591.770685},{"request_id":"1357132:1:1357132:516","session_id":"1357132","input_length":13520,"t_proxy_recv":348725.423223203,"t_decision_unix":1779779590.4371643,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":13520,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":13520,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":122597,"ongoing_decode_tokens":0,"pending_prefill_tokens":121573,"num_requests":1,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":13520,"score_linear":122597.0,"score_lmetric":135093},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":968,"cache_hit":0,"new_prefill":13520,"score_linear":62618.0,"score_lmetric":13520},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":35308,"ongoing_decode_tokens":0,"pending_prefill_tokens":35308,"num_requests":1,"active_p_offloads":0,"cached_blocks":1398,"cache_hit":0,"new_prefill":13520,"score_linear":35308.0,"score_lmetric":48828},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":134962,"ongoing_decode_tokens":0,"pending_prefill_tokens":83762,"num_requests":2,"active_p_offloads":0,"cached_blocks":1294,"cache_hit":0,"new_prefill":13520,"score_linear":134962.0,"score_lmetric":194564},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121673,"ongoing_decode_tokens":24503,"pending_prefill_tokens":3986,"num_requests":2,"active_p_offloads":0,"cached_blocks":1882,"cache_hit":0,"new_prefill":13520,"score_linear":121673.0,"score_lmetric":35012},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1129,"cache_hit":0,"new_prefill":13520,"score_linear":68259.0,"score_lmetric":13520},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":6711,"ongoing_decode_tokens":6711,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1045,"cache_hit":0,"new_prefill":13520,"score_linear":6711.0,"score_lmetric":13520},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":995,"cache_hit":0,"new_prefill":13520,"score_linear":21371.0,"score_lmetric":13520}],"chosen_score_linear":62618.0,"chosen_score_lmetric":13520,"t_first_token":348726.594620121,"t_first_token_unix":1779779591.6085637,"t_done":348727.746153495,"t_done_unix":1779779592.7600965},{"request_id":"1344773:7:1357714:518","session_id":"1344773","input_length":34351,"t_proxy_recv":348727.355531035,"t_decision_unix":1779779592.3694725,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":33792,"affinity_cache_ratio":0.9837268201799074,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":33792,"estimated_new_tokens":559,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":122597,"ongoing_decode_tokens":0,"pending_prefill_tokens":121573,"num_requests":1,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":34351,"score_linear":122597.0,"score_lmetric":155924},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":76138,"ongoing_decode_tokens":76138,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":968,"cache_hit":0,"new_prefill":34351,"score_linear":76138.0,"score_lmetric":68702},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1467,"cache_hit":0,"new_prefill":34351,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":134962,"ongoing_decode_tokens":0,"pending_prefill_tokens":83762,"num_requests":2,"active_p_offloads":0,"cached_blocks":1294,"cache_hit":0,"new_prefill":34351,"score_linear":134962.0,"score_lmetric":236226},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":149478,"ongoing_decode_tokens":24503,"pending_prefill_tokens":4143,"num_requests":3,"active_p_offloads":0,"cached_blocks":1882,"cache_hit":0,"new_prefill":34351,"score_linear":149478.0,"score_lmetric":115482},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1129,"cache_hit":33792,"new_prefill":559,"score_linear":34467.0,"score_lmetric":559},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1058,"cache_hit":0,"new_prefill":34351,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":995,"cache_hit":0,"new_prefill":34351,"score_linear":21371.0,"score_lmetric":34351}],"chosen_score_linear":34467.0,"chosen_score_lmetric":559,"t_first_token":348727.484165765,"t_first_token_unix":1779779592.4981093,"t_done":348728.146040001,"t_done_unix":1779779593.1599844},{"request_id":"1355325:1:1355325:507","session_id":"1355325","input_length":24503,"t_proxy_recv":348719.165165391,"t_decision_unix":1779779584.179106,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":-24503,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":24503,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1131,"cache_hit":0,"new_prefill":24503,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":968,"cache_hit":0,"new_prefill":24503,"score_linear":62618.0,"score_lmetric":24503},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1398,"cache_hit":0,"new_prefill":24503,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":134962,"ongoing_decode_tokens":0,"pending_prefill_tokens":83762,"num_requests":2,"active_p_offloads":0,"cached_blocks":1294,"cache_hit":0,"new_prefill":24503,"score_linear":134962.0,"score_lmetric":216530},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1882,"cache_hit":0,"new_prefill":24503,"score_linear":0.0,"score_lmetric":-24503},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":83778,"ongoing_decode_tokens":83778,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1126,"cache_hit":0,"new_prefill":24503,"score_linear":83778.0,"score_lmetric":49006},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1045,"cache_hit":0,"new_prefill":24503,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":994,"cache_hit":0,"new_prefill":24503,"score_linear":21371.0,"score_lmetric":24503}],"chosen_score_linear":0.0,"chosen_score_lmetric":-24503,"t_first_token":348721.998431821,"t_first_token_unix":1779779587.0123756,"t_done":348730.997873848,"t_done_unix":1779779596.0118172},{"request_id":"1253804:6:1318560:340","session_id":"1253804","input_length":62618,"t_proxy_recv":348589.511813541,"t_decision_unix":1779779454.5257547,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":58880,"affinity_cache_ratio":0.9403047047174934,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":58880,"estimated_new_tokens":3738,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":739,"cache_hit":0,"new_prefill":62618,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":947,"cache_hit":58880,"new_prefill":3738,"score_linear":-58880.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":908,"cache_hit":0,"new_prefill":62618,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":911,"cache_hit":3584,"new_prefill":59034,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":134853,"ongoing_decode_tokens":102413,"pending_prefill_tokens":14520,"num_requests":2,"active_p_offloads":0,"cached_blocks":1257,"cache_hit":0,"new_prefill":62618,"score_linear":134853.0,"score_lmetric":154276},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":104431,"ongoing_decode_tokens":0,"pending_prefill_tokens":80367,"num_requests":2,"active_p_offloads":0,"cached_blocks":655,"cache_hit":3584,"new_prefill":59034,"score_linear":100847.0,"score_lmetric":278802},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":820,"cache_hit":0,"new_prefill":62618,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":176419,"ongoing_decode_tokens":176419,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":810,"cache_hit":0,"new_prefill":62618,"score_linear":176419.0,"score_lmetric":125236}],"chosen_score_linear":-58880.0,"chosen_score_lmetric":0,"t_first_token":348599.659938111,"t_first_token_unix":1779779464.6738818,"t_done":348731.37336342,"t_done_unix":1779779596.3873072},{"request_id":"1340278:2:1356079:512","session_id":"1340278","input_length":48977,"t_proxy_recv":348721.794052033,"t_decision_unix":1779779586.807993,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":48128,"affinity_cache_ratio":0.9826653327071891,"affinity_num_requests":1,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":48128,"estimated_new_tokens":849,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":48977,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":968,"cache_hit":0,"new_prefill":48977,"score_linear":62618.0,"score_lmetric":48977},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":35308,"ongoing_decode_tokens":0,"pending_prefill_tokens":35308,"num_requests":1,"active_p_offloads":0,"cached_blocks":1398,"cache_hit":0,"new_prefill":48977,"score_linear":35308.0,"score_lmetric":84285},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":134962,"ongoing_decode_tokens":0,"pending_prefill_tokens":83762,"num_requests":2,"active_p_offloads":0,"cached_blocks":1294,"cache_hit":0,"new_prefill":48977,"score_linear":134962.0,"score_lmetric":265478},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":72696,"ongoing_decode_tokens":0,"pending_prefill_tokens":27640,"num_requests":1,"active_p_offloads":0,"cached_blocks":1882,"cache_hit":48128,"new_prefill":849,"score_linear":24568.0,"score_lmetric":28489},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":83778,"ongoing_decode_tokens":83778,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1126,"cache_hit":0,"new_prefill":48977,"score_linear":83778.0,"score_lmetric":97954},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1045,"cache_hit":0,"new_prefill":48977,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":99460,"ongoing_decode_tokens":99460,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":994,"cache_hit":0,"new_prefill":48977,"score_linear":99460.0,"score_lmetric":97954}],"chosen_score_linear":24568.0,"chosen_score_lmetric":28489,"t_first_token":348730.973016818,"t_first_token_unix":1779779595.9869604,"t_done":348732.186118808,"t_done_unix":1779779597.2000623},{"request_id":"1344773:8:1358989:523","session_id":"1344773","input_length":34552,"t_proxy_recv":348731.881084543,"t_decision_unix":1779779596.8950262,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":34304,"affinity_cache_ratio":0.9928224125955082,"affinity_num_requests":1,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":34304,"estimated_new_tokens":248,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":122597,"ongoing_decode_tokens":0,"pending_prefill_tokens":121573,"num_requests":1,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":34552,"score_linear":122597.0,"score_lmetric":156125},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":34552,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1467,"cache_hit":0,"new_prefill":34552,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":134962,"ongoing_decode_tokens":0,"pending_prefill_tokens":83762,"num_requests":2,"active_p_offloads":0,"cached_blocks":1294,"cache_hit":0,"new_prefill":34552,"score_linear":134962.0,"score_lmetric":236628},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":124975,"ongoing_decode_tokens":124975,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1929,"cache_hit":0,"new_prefill":34552,"score_linear":124975.0,"score_lmetric":69104},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1130,"cache_hit":34304,"new_prefill":248,"score_linear":33955.0,"score_lmetric":248},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":58758,"ongoing_decode_tokens":0,"pending_prefill_tokens":45958,"num_requests":1,"active_p_offloads":0,"cached_blocks":1058,"cache_hit":0,"new_prefill":34552,"score_linear":58758.0,"score_lmetric":80510},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":101617,"ongoing_decode_tokens":21371,"pending_prefill_tokens":1910,"num_requests":2,"active_p_offloads":0,"cached_blocks":995,"cache_hit":0,"new_prefill":34552,"score_linear":101617.0,"score_lmetric":72924}],"chosen_score_linear":33955.0,"chosen_score_lmetric":248,"t_first_token":348732.017322195,"t_first_token_unix":1779779597.031266,"t_done":348732.445878768,"t_done_unix":1779779597.4598222},{"request_id":"1353821:2:1357420:517","session_id":"1353821","input_length":27805,"t_proxy_recv":348726.275093532,"t_decision_unix":1779779591.2890317,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":27648,"affinity_cache_ratio":0.9943535335371336,"affinity_num_requests":2,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":27648,"estimated_new_tokens":157,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":122597,"ongoing_decode_tokens":0,"pending_prefill_tokens":121573,"num_requests":1,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":27805,"score_linear":122597.0,"score_lmetric":149378},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":76138,"ongoing_decode_tokens":62618,"pending_prefill_tokens":13520,"num_requests":2,"active_p_offloads":0,"cached_blocks":968,"cache_hit":0,"new_prefill":27805,"score_linear":76138.0,"score_lmetric":82650},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":35308,"ongoing_decode_tokens":0,"pending_prefill_tokens":35308,"num_requests":1,"active_p_offloads":0,"cached_blocks":1398,"cache_hit":0,"new_prefill":27805,"score_linear":35308.0,"score_lmetric":63113},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":134962,"ongoing_decode_tokens":0,"pending_prefill_tokens":83762,"num_requests":2,"active_p_offloads":0,"cached_blocks":1294,"cache_hit":0,"new_prefill":27805,"score_linear":134962.0,"score_lmetric":223134},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121673,"ongoing_decode_tokens":24503,"pending_prefill_tokens":3986,"num_requests":2,"active_p_offloads":0,"cached_blocks":1882,"cache_hit":27648,"new_prefill":157,"score_linear":94025.0,"score_lmetric":8286},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1129,"cache_hit":0,"new_prefill":27805,"score_linear":68259.0,"score_lmetric":27805},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1058,"cache_hit":0,"new_prefill":27805,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":995,"cache_hit":0,"new_prefill":27805,"score_linear":21371.0,"score_lmetric":27805}],"chosen_score_linear":94025.0,"chosen_score_lmetric":8286,"t_first_token":348730.973293432,"t_first_token_unix":1779779595.9872365,"t_done":348732.460027062,"t_done_unix":1779779597.4739704},{"request_id":"1344773:9:1360064:526","session_id":"1344773","input_length":34611,"t_proxy_recv":348735.881419185,"t_decision_unix":1779779600.8953602,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":34304,"affinity_cache_ratio":0.9911299875762041,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":34304,"estimated_new_tokens":307,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":142090,"ongoing_decode_tokens":0,"pending_prefill_tokens":121610,"num_requests":2,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":34611,"score_linear":142090.0,"score_lmetric":312442},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":34611,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1467,"cache_hit":0,"new_prefill":34611,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":134962,"ongoing_decode_tokens":0,"pending_prefill_tokens":83762,"num_requests":2,"active_p_offloads":0,"cached_blocks":1294,"cache_hit":0,"new_prefill":34611,"score_linear":134962.0,"score_lmetric":236746},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":48193,"ongoing_decode_tokens":48193,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1930,"cache_hit":0,"new_prefill":34611,"score_linear":48193.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1130,"cache_hit":34304,"new_prefill":307,"score_linear":33955.0,"score_lmetric":307},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":58758,"ongoing_decode_tokens":0,"pending_prefill_tokens":45958,"num_requests":1,"active_p_offloads":0,"cached_blocks":1058,"cache_hit":0,"new_prefill":34611,"score_linear":58758.0,"score_lmetric":80569},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":101617,"ongoing_decode_tokens":101617,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":995,"cache_hit":0,"new_prefill":34611,"score_linear":101617.0,"score_lmetric":69222}],"chosen_score_linear":33955.0,"chosen_score_lmetric":307,"t_first_token":348735.958375392,"t_first_token_unix":1779779600.9723184,"t_done":348736.627357958,"t_done_unix":1779779601.6413014},{"request_id":"1253743:6:1354346:505","session_id":"1253743","input_length":51436,"t_proxy_recv":348715.724560828,"t_decision_unix":1779779580.7385018,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":51200,"affinity_cache_ratio":0.9954117738548877,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":51200,"estimated_new_tokens":236,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1131,"cache_hit":0,"new_prefill":51436,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":968,"cache_hit":4608,"new_prefill":46828,"score_linear":58010.0,"score_lmetric":46828},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1398,"cache_hit":0,"new_prefill":51436,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":83526,"ongoing_decode_tokens":0,"pending_prefill_tokens":83526,"num_requests":1,"active_p_offloads":0,"cached_blocks":1294,"cache_hit":51200,"new_prefill":236,"score_linear":32326.0,"score_lmetric":83762},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27651,"ongoing_decode_tokens":0,"pending_prefill_tokens":27651,"num_requests":0,"active_p_offloads":0,"cached_blocks":1832,"cache_hit":0,"new_prefill":51436,"score_linear":27651.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":102134,"ongoing_decode_tokens":102134,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1125,"cache_hit":17408,"new_prefill":34028,"score_linear":84726.0,"score_lmetric":68056},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1045,"cache_hit":0,"new_prefill":51436,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":994,"cache_hit":0,"new_prefill":51436,"score_linear":21371.0,"score_lmetric":51436}],"chosen_score_linear":32326.0,"chosen_score_lmetric":83762,"t_first_token":348737.918693712,"t_first_token_unix":1779779602.932637,"t_done":348738.596459017,"t_done_unix":1779779603.6104023},{"request_id":"1253743:7:1359251:524","session_id":"1253743","input_length":51509,"t_proxy_recv":348738.604790377,"t_decision_unix":1779779603.618732,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":51200,"affinity_cache_ratio":0.9940010483604806,"affinity_num_requests":1,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":51200,"estimated_new_tokens":309,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":142090,"ongoing_decode_tokens":0,"pending_prefill_tokens":121610,"num_requests":2,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":51509,"score_linear":142090.0,"score_lmetric":346238},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":117888,"ongoing_decode_tokens":0,"pending_prefill_tokens":117888,"num_requests":1,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":4608,"new_prefill":46901,"score_linear":113280.0,"score_lmetric":164789},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1467,"cache_hit":0,"new_prefill":51509,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":83526,"ongoing_decode_tokens":83526,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1294,"cache_hit":51200,"new_prefill":309,"score_linear":32326.0,"score_lmetric":309},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":81671,"ongoing_decode_tokens":48193,"pending_prefill_tokens":5318,"num_requests":1,"active_p_offloads":0,"cached_blocks":1930,"cache_hit":0,"new_prefill":51509,"score_linear":81671.0,"score_lmetric":56827},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1130,"cache_hit":17408,"new_prefill":34101,"score_linear":50851.0,"score_lmetric":34101},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":58758,"ongoing_decode_tokens":0,"pending_prefill_tokens":45958,"num_requests":1,"active_p_offloads":0,"cached_blocks":1058,"cache_hit":0,"new_prefill":51509,"score_linear":58758.0,"score_lmetric":97467},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":101617,"ongoing_decode_tokens":101617,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":995,"cache_hit":0,"new_prefill":51509,"score_linear":101617.0,"score_lmetric":103018}],"chosen_score_linear":32326.0,"chosen_score_lmetric":309,"t_first_token":348738.728760716,"t_first_token_unix":1779779603.7427037,"t_done":348739.019504047,"t_done_unix":1779779604.0334477},{"request_id":"1353951:1:1353951:504","session_id":"1353951","input_length":83526,"t_proxy_recv":348714.371523678,"t_decision_unix":1779779579.385465,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":83526,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":96026,"ongoing_decode_tokens":96026,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1129,"cache_hit":0,"new_prefill":83526,"score_linear":96026.0,"score_lmetric":83526},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":968,"cache_hit":0,"new_prefill":83526,"score_linear":62618.0,"score_lmetric":83526},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1398,"cache_hit":0,"new_prefill":83526,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1294,"cache_hit":0,"new_prefill":83526,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27651,"ongoing_decode_tokens":0,"pending_prefill_tokens":27651,"num_requests":0,"active_p_offloads":0,"cached_blocks":1832,"cache_hit":0,"new_prefill":83526,"score_linear":27651.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":102134,"ongoing_decode_tokens":102134,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1125,"cache_hit":0,"new_prefill":83526,"score_linear":102134.0,"score_lmetric":167052},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1045,"cache_hit":0,"new_prefill":83526,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":994,"cache_hit":0,"new_prefill":83526,"score_linear":21371.0,"score_lmetric":83526}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348737.918150106,"t_first_token_unix":1779779602.932096,"t_done":348740.011843869,"t_done_unix":1779779605.0257874},{"request_id":"1342921:2:1361022:530","session_id":"1342921","input_length":66032,"t_proxy_recv":348739.156053625,"t_decision_unix":1779779604.169995,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":65536,"affinity_cache_ratio":0.992488490428883,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":65536,"estimated_new_tokens":496,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":142090,"ongoing_decode_tokens":0,"pending_prefill_tokens":121610,"num_requests":2,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":66032,"score_linear":142090.0,"score_lmetric":375284},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":117888,"ongoing_decode_tokens":0,"pending_prefill_tokens":117888,"num_requests":1,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":66032,"score_linear":117888.0,"score_lmetric":183920},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1467,"cache_hit":0,"new_prefill":66032,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":83526,"ongoing_decode_tokens":83526,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1294,"cache_hit":65536,"new_prefill":496,"score_linear":17990.0,"score_lmetric":496},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":109604,"ongoing_decode_tokens":48193,"pending_prefill_tokens":5603,"num_requests":2,"active_p_offloads":0,"cached_blocks":1930,"cache_hit":0,"new_prefill":66032,"score_linear":109604.0,"score_lmetric":143270},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1130,"cache_hit":0,"new_prefill":66032,"score_linear":68259.0,"score_lmetric":66032},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":58758,"ongoing_decode_tokens":0,"pending_prefill_tokens":45958,"num_requests":1,"active_p_offloads":0,"cached_blocks":1058,"cache_hit":0,"new_prefill":66032,"score_linear":58758.0,"score_lmetric":111990},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":101617,"ongoing_decode_tokens":101617,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":995,"cache_hit":0,"new_prefill":66032,"score_linear":101617.0,"score_lmetric":132064}],"chosen_score_linear":17990.0,"chosen_score_lmetric":496,"t_first_token":348739.517496478,"t_first_token_unix":1779779604.5314405,"t_done":348740.300097645,"t_done_unix":1779779605.314041},{"request_id":"1353951:2:1358300:520","session_id":"1353951","input_length":83905,"t_proxy_recv":348740.027883994,"t_decision_unix":1779779605.0418255,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":83456,"affinity_cache_ratio":0.9946487098504261,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":83456,"estimated_new_tokens":449,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":142090,"ongoing_decode_tokens":0,"pending_prefill_tokens":121610,"num_requests":2,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":83905,"score_linear":142090.0,"score_lmetric":411030},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":117888,"ongoing_decode_tokens":0,"pending_prefill_tokens":117888,"num_requests":1,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":83905,"score_linear":117888.0,"score_lmetric":201793},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1467,"cache_hit":0,"new_prefill":83905,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":66032,"ongoing_decode_tokens":66032,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1457,"cache_hit":83456,"new_prefill":449,"score_linear":-17424.0,"score_lmetric":449},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":109604,"ongoing_decode_tokens":48193,"pending_prefill_tokens":5603,"num_requests":2,"active_p_offloads":0,"cached_blocks":1930,"cache_hit":0,"new_prefill":83905,"score_linear":109604.0,"score_lmetric":179016},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1130,"cache_hit":0,"new_prefill":83905,"score_linear":68259.0,"score_lmetric":83905},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":58758,"ongoing_decode_tokens":0,"pending_prefill_tokens":45958,"num_requests":1,"active_p_offloads":0,"cached_blocks":1058,"cache_hit":0,"new_prefill":83905,"score_linear":58758.0,"score_lmetric":129863},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":101617,"ongoing_decode_tokens":101617,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":995,"cache_hit":0,"new_prefill":83905,"score_linear":101617.0,"score_lmetric":167810}],"chosen_score_linear":-17424.0,"chosen_score_lmetric":449,"t_first_token":348740.295831493,"t_first_token_unix":1779779605.3097787,"t_done":348740.44909537,"t_done_unix":1779779605.4630387},{"request_id":"1356948:2:1361262:532","session_id":"1356948","input_length":16480,"t_proxy_recv":348740.168350319,"t_decision_unix":1779779605.1822915,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":6,"chosen_idx":2,"affinity_cache_hit":6656,"affinity_cache_ratio":0.40388349514563104,"affinity_num_requests":1,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":16480,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":142090,"ongoing_decode_tokens":0,"pending_prefill_tokens":121610,"num_requests":2,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":16480,"score_linear":142090.0,"score_lmetric":276180},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":117888,"ongoing_decode_tokens":0,"pending_prefill_tokens":117888,"num_requests":1,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":16480,"score_linear":117888.0,"score_lmetric":134368},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1467,"cache_hit":0,"new_prefill":16480,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":149937,"ongoing_decode_tokens":66032,"pending_prefill_tokens":449,"num_requests":2,"active_p_offloads":0,"cached_blocks":1457,"cache_hit":0,"new_prefill":16480,"score_linear":149937.0,"score_lmetric":33858},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":109604,"ongoing_decode_tokens":48193,"pending_prefill_tokens":5603,"num_requests":2,"active_p_offloads":0,"cached_blocks":1930,"cache_hit":0,"new_prefill":16480,"score_linear":109604.0,"score_lmetric":44166},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1130,"cache_hit":0,"new_prefill":16480,"score_linear":68259.0,"score_lmetric":16480},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":58758,"ongoing_decode_tokens":58758,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1058,"cache_hit":6656,"new_prefill":9824,"score_linear":52102.0,"score_lmetric":9824},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":101617,"ongoing_decode_tokens":101617,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":995,"cache_hit":0,"new_prefill":16480,"score_linear":101617.0,"score_lmetric":32960}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348741.709799319,"t_first_token_unix":1779779606.7237992,"t_done":348741.876866214,"t_done_unix":1779779606.8908093},{"request_id":"1336620:3:1339999:443","session_id":"1336620","input_length":21371,"t_proxy_recv":348665.775309475,"t_decision_unix":1779779530.7892509,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":14848,"affinity_cache_ratio":0.6947732909082401,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":14848,"estimated_new_tokens":6523,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":176906,"ongoing_decode_tokens":81523,"pending_prefill_tokens":8343,"num_requests":3,"active_p_offloads":0,"cached_blocks":893,"cache_hit":7680,"new_prefill":13691,"score_linear":169226.0,"score_lmetric":66102},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":963,"cache_hit":0,"new_prefill":21371,"score_linear":62618.0,"score_lmetric":21371},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":45994,"ongoing_decode_tokens":45994,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1238,"cache_hit":0,"new_prefill":21371,"score_linear":45994.0,"score_lmetric":21371},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":13816,"ongoing_decode_tokens":0,"pending_prefill_tokens":13816,"num_requests":1,"active_p_offloads":0,"cached_blocks":1026,"cache_hit":0,"new_prefill":21371,"score_linear":13816.0,"score_lmetric":35187},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":55130,"ongoing_decode_tokens":55130,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1643,"cache_hit":7680,"new_prefill":13691,"score_linear":47450.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":66152,"ongoing_decode_tokens":0,"pending_prefill_tokens":4712,"num_requests":1,"active_p_offloads":0,"cached_blocks":1019,"cache_hit":0,"new_prefill":21371,"score_linear":66152.0,"score_lmetric":26083},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":31527,"ongoing_decode_tokens":31527,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":988,"cache_hit":7680,"new_prefill":13691,"score_linear":23847.0,"score_lmetric":27382},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":917,"cache_hit":14848,"new_prefill":6523,"score_linear":-14848.0,"score_lmetric":0}],"chosen_score_linear":-14848.0,"chosen_score_lmetric":0,"t_first_token":348666.746129004,"t_first_token_unix":1779779531.7600727,"t_done":348742.716995314,"t_done_unix":1779779607.7309396},{"request_id":"1361992:1:1361992:535","session_id":"1361992","input_length":8113,"t_proxy_recv":348742.744488548,"t_decision_unix":1779779607.7584302,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":8113,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":142090,"ongoing_decode_tokens":0,"pending_prefill_tokens":121610,"num_requests":2,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":8113,"score_linear":142090.0,"score_lmetric":259446},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":117888,"ongoing_decode_tokens":0,"pending_prefill_tokens":117888,"num_requests":1,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":8113,"score_linear":117888.0,"score_lmetric":126001},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1499,"cache_hit":0,"new_prefill":8113,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76969,"ongoing_decode_tokens":0,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1458,"cache_hit":0,"new_prefill":8113,"score_linear":76969.0,"score_lmetric":85082},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":109604,"ongoing_decode_tokens":48193,"pending_prefill_tokens":5603,"num_requests":2,"active_p_offloads":0,"cached_blocks":1930,"cache_hit":0,"new_prefill":8113,"score_linear":109604.0,"score_lmetric":27432},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":102938,"ongoing_decode_tokens":102938,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1130,"cache_hit":0,"new_prefill":8113,"score_linear":102938.0,"score_lmetric":16226},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":58758,"ongoing_decode_tokens":58758,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1058,"cache_hit":0,"new_prefill":8113,"score_linear":58758.0,"score_lmetric":8113},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1011,"cache_hit":0,"new_prefill":8113,"score_linear":80246.0,"score_lmetric":8113}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348743.312686386,"t_first_token_unix":1779779608.3266308,"t_done":348743.449488969,"t_done_unix":1779779608.463432},{"request_id":"1353821:3:1360898:529","session_id":"1353821","input_length":27933,"t_proxy_recv":348738.692643417,"t_decision_unix":1779779603.7065847,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":27648,"affinity_cache_ratio":0.98979701428418,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":27648,"estimated_new_tokens":285,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":142090,"ongoing_decode_tokens":0,"pending_prefill_tokens":121610,"num_requests":2,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":27933,"score_linear":142090.0,"score_lmetric":299086},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":117888,"ongoing_decode_tokens":0,"pending_prefill_tokens":117888,"num_requests":1,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":27933,"score_linear":117888.0,"score_lmetric":145821},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1467,"cache_hit":0,"new_prefill":27933,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":135035,"ongoing_decode_tokens":83526,"pending_prefill_tokens":309,"num_requests":2,"active_p_offloads":0,"cached_blocks":1294,"cache_hit":0,"new_prefill":27933,"score_linear":135035.0,"score_lmetric":56484},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":81671,"ongoing_decode_tokens":48193,"pending_prefill_tokens":5318,"num_requests":1,"active_p_offloads":0,"cached_blocks":1930,"cache_hit":27648,"new_prefill":285,"score_linear":54023.0,"score_lmetric":5603},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1130,"cache_hit":0,"new_prefill":27933,"score_linear":68259.0,"score_lmetric":27933},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":58758,"ongoing_decode_tokens":0,"pending_prefill_tokens":45958,"num_requests":1,"active_p_offloads":0,"cached_blocks":1058,"cache_hit":0,"new_prefill":27933,"score_linear":58758.0,"score_lmetric":73891},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":101617,"ongoing_decode_tokens":101617,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":995,"cache_hit":0,"new_prefill":27933,"score_linear":101617.0,"score_lmetric":55866}],"chosen_score_linear":54023.0,"chosen_score_lmetric":5603,"t_first_token":348742.844446426,"t_first_token_unix":1779779607.85839,"t_done":348743.650300774,"t_done_unix":1779779608.6642442},{"request_id":"1270340:3:1360716:528","session_id":"1270340","input_length":33478,"t_proxy_recv":348738.026689795,"t_decision_unix":1779779603.040631,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":28160,"affinity_cache_ratio":0.8411494115538563,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":28160,"estimated_new_tokens":5318,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":142090,"ongoing_decode_tokens":0,"pending_prefill_tokens":121610,"num_requests":2,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":33478,"score_linear":142090.0,"score_lmetric":310176},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":117888,"ongoing_decode_tokens":0,"pending_prefill_tokens":117888,"num_requests":1,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":33478,"score_linear":117888.0,"score_lmetric":151366},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1467,"cache_hit":0,"new_prefill":33478,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":134962,"ongoing_decode_tokens":134962,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1294,"cache_hit":1536,"new_prefill":31942,"score_linear":133426.0,"score_lmetric":63884},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":48193,"ongoing_decode_tokens":48193,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1930,"cache_hit":28160,"new_prefill":5318,"score_linear":20033.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1130,"cache_hit":0,"new_prefill":33478,"score_linear":68259.0,"score_lmetric":33478},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":58758,"ongoing_decode_tokens":0,"pending_prefill_tokens":45958,"num_requests":1,"active_p_offloads":0,"cached_blocks":1058,"cache_hit":0,"new_prefill":33478,"score_linear":58758.0,"score_lmetric":79436},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":101617,"ongoing_decode_tokens":101617,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":995,"cache_hit":0,"new_prefill":33478,"score_linear":101617.0,"score_lmetric":66956}],"chosen_score_linear":20033.0,"chosen_score_lmetric":0,"t_first_token":348742.840086914,"t_first_token_unix":1779779607.8540306,"t_done":348743.794185068,"t_done_unix":1779779608.8081284},{"request_id":"1294611:11:1355750:510","session_id":"1294611","input_length":48193,"t_proxy_recv":348720.376082301,"t_decision_unix":1779779585.3900237,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":45056,"affinity_cache_ratio":0.9349075591890939,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":45056,"estimated_new_tokens":3137,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":19366,"ongoing_decode_tokens":0,"pending_prefill_tokens":6566,"num_requests":1,"active_p_offloads":0,"cached_blocks":1131,"cache_hit":0,"new_prefill":48193,"score_linear":19366.0,"score_lmetric":54759},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":968,"cache_hit":0,"new_prefill":48193,"score_linear":62618.0,"score_lmetric":48193},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1398,"cache_hit":0,"new_prefill":48193,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":134962,"ongoing_decode_tokens":0,"pending_prefill_tokens":83762,"num_requests":2,"active_p_offloads":0,"cached_blocks":1294,"cache_hit":0,"new_prefill":48193,"score_linear":134962.0,"score_lmetric":263910},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":24503,"ongoing_decode_tokens":0,"pending_prefill_tokens":24503,"num_requests":0,"active_p_offloads":0,"cached_blocks":1882,"cache_hit":45056,"new_prefill":3137,"score_linear":-20553.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":83778,"ongoing_decode_tokens":83778,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1126,"cache_hit":12800,"new_prefill":35393,"score_linear":70978.0,"score_lmetric":70786},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1045,"cache_hit":0,"new_prefill":48193,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":99460,"ongoing_decode_tokens":21371,"pending_prefill_tokens":265,"num_requests":2,"active_p_offloads":0,"cached_blocks":994,"cache_hit":12800,"new_prefill":35393,"score_linear":86660.0,"score_lmetric":71316}],"chosen_score_linear":-20553.0,"chosen_score_lmetric":0,"t_first_token":348730.968265881,"t_first_token_unix":1779779595.9822104,"t_done":348744.190079567,"t_done_unix":1779779609.2040231},{"request_id":"1344773:10:1361326:533","session_id":"1344773","input_length":34679,"t_proxy_recv":348740.411430019,"t_decision_unix":1779779605.4253714,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":34304,"affinity_cache_ratio":0.9891865394042504,"affinity_num_requests":1,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":34304,"estimated_new_tokens":375,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":142090,"ongoing_decode_tokens":0,"pending_prefill_tokens":121610,"num_requests":2,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":34679,"score_linear":142090.0,"score_lmetric":312578},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":117888,"ongoing_decode_tokens":0,"pending_prefill_tokens":117888,"num_requests":1,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":34679,"score_linear":117888.0,"score_lmetric":152567},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":16480,"ongoing_decode_tokens":0,"pending_prefill_tokens":16480,"num_requests":1,"active_p_offloads":0,"cached_blocks":1467,"cache_hit":0,"new_prefill":34679,"score_linear":16480.0,"score_lmetric":51159},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":83905,"ongoing_decode_tokens":83905,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1458,"cache_hit":0,"new_prefill":34679,"score_linear":83905.0,"score_lmetric":34679},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":109604,"ongoing_decode_tokens":48193,"pending_prefill_tokens":5603,"num_requests":2,"active_p_offloads":0,"cached_blocks":1930,"cache_hit":0,"new_prefill":34679,"score_linear":109604.0,"score_lmetric":80564},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1130,"cache_hit":34304,"new_prefill":375,"score_linear":33955.0,"score_lmetric":375},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":58758,"ongoing_decode_tokens":58758,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1058,"cache_hit":0,"new_prefill":34679,"score_linear":58758.0,"score_lmetric":34679},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":101617,"ongoing_decode_tokens":101617,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":995,"cache_hit":0,"new_prefill":34679,"score_linear":101617.0,"score_lmetric":69358}],"chosen_score_linear":33955.0,"chosen_score_lmetric":375,"t_first_token":348740.488486712,"t_first_token_unix":1779779605.50243,"t_done":348744.587059808,"t_done_unix":1779779609.6010034},{"request_id":"1362265:1:1362265:536","session_id":"1362265","input_length":8825,"t_proxy_recv":348743.591584561,"t_decision_unix":1779779608.605526,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":8825,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":142090,"ongoing_decode_tokens":0,"pending_prefill_tokens":121610,"num_requests":2,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":8825,"score_linear":142090.0,"score_lmetric":260870},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":117888,"ongoing_decode_tokens":0,"pending_prefill_tokens":117888,"num_requests":1,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":8825,"score_linear":117888.0,"score_lmetric":126713},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1514,"cache_hit":0,"new_prefill":8825,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76969,"ongoing_decode_tokens":0,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1458,"cache_hit":0,"new_prefill":8825,"score_linear":76969.0,"score_lmetric":85794},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":109604,"ongoing_decode_tokens":109604,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1930,"cache_hit":0,"new_prefill":8825,"score_linear":109604.0,"score_lmetric":17650},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":102938,"ongoing_decode_tokens":102938,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1130,"cache_hit":0,"new_prefill":8825,"score_linear":102938.0,"score_lmetric":17650},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":58758,"ongoing_decode_tokens":58758,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1058,"cache_hit":0,"new_prefill":8825,"score_linear":58758.0,"score_lmetric":8825},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1011,"cache_hit":0,"new_prefill":8825,"score_linear":80246.0,"score_lmetric":8825}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348744.252587742,"t_first_token_unix":1779779609.266531,"t_done":348745.32052191,"t_done_unix":1779779610.334465},{"request_id":"1313181:2:1358046:519","session_id":"1313181","input_length":58758,"t_proxy_recv":348728.558405263,"t_decision_unix":1779779593.5723464,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":0,"chosen_idx":6,"affinity_cache_hit":24064,"affinity_cache_ratio":0.40954423227475406,"affinity_num_requests":1,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":false,"cache_hit":12800,"estimated_new_tokens":45958,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":122597,"ongoing_decode_tokens":0,"pending_prefill_tokens":121573,"num_requests":1,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":24064,"new_prefill":34694,"score_linear":98533.0,"score_lmetric":156267},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":994,"cache_hit":0,"new_prefill":58758,"score_linear":62618.0,"score_lmetric":58758},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1467,"cache_hit":0,"new_prefill":58758,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":134962,"ongoing_decode_tokens":0,"pending_prefill_tokens":83762,"num_requests":2,"active_p_offloads":0,"cached_blocks":1294,"cache_hit":0,"new_prefill":58758,"score_linear":134962.0,"score_lmetric":285040},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":149478,"ongoing_decode_tokens":24503,"pending_prefill_tokens":4143,"num_requests":3,"active_p_offloads":0,"cached_blocks":1882,"cache_hit":0,"new_prefill":58758,"score_linear":149478.0,"score_lmetric":188703},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1130,"cache_hit":0,"new_prefill":58758,"score_linear":68259.0,"score_lmetric":58758},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1058,"cache_hit":12800,"new_prefill":45958,"score_linear":-12800.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":995,"cache_hit":12800,"new_prefill":45958,"score_linear":8571.0,"score_lmetric":45958}],"chosen_score_linear":-12800.0,"chosen_score_lmetric":0,"t_first_token":348740.107355437,"t_first_token_unix":1779779605.121299,"t_done":348746.031739938,"t_done_unix":1779779611.0456834},{"request_id":"1361992:2:1362904:537","session_id":"1361992","input_length":10347,"t_proxy_recv":348745.835637102,"t_decision_unix":1779779610.8495784,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":7680,"affinity_cache_ratio":0.7422441287329661,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":7680,"estimated_new_tokens":2667,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":142090,"ongoing_decode_tokens":0,"pending_prefill_tokens":121610,"num_requests":2,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":10347,"score_linear":142090.0,"score_lmetric":263914},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":117888,"ongoing_decode_tokens":0,"pending_prefill_tokens":117888,"num_requests":1,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":10347,"score_linear":117888.0,"score_lmetric":128235},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1531,"cache_hit":7680,"new_prefill":2667,"score_linear":-7680.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76969,"ongoing_decode_tokens":0,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1458,"cache_hit":0,"new_prefill":10347,"score_linear":76969.0,"score_lmetric":87316},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1948,"cache_hit":0,"new_prefill":10347,"score_linear":0.0,"score_lmetric":-10347},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1131,"cache_hit":0,"new_prefill":10347,"score_linear":68259.0,"score_lmetric":10347},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":58758,"ongoing_decode_tokens":58758,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1058,"cache_hit":0,"new_prefill":10347,"score_linear":58758.0,"score_lmetric":10347},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1011,"cache_hit":0,"new_prefill":10347,"score_linear":80246.0,"score_lmetric":10347}],"chosen_score_linear":-7680.0,"chosen_score_lmetric":0,"t_first_token":348746.096442875,"t_first_token_unix":1779779611.1103864,"t_done":348746.243665741,"t_done_unix":1779779611.2576091},{"request_id":"1313181:3:1316742:330","session_id":"1313181","input_length":24451,"t_proxy_recv":348746.036659065,"t_decision_unix":1779779611.0506008,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":24064,"affinity_cache_ratio":0.9841724264856243,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":24064,"estimated_new_tokens":387,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":142090,"ongoing_decode_tokens":0,"pending_prefill_tokens":121610,"num_requests":2,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":24064,"new_prefill":387,"score_linear":118026.0,"score_lmetric":243994},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":117888,"ongoing_decode_tokens":0,"pending_prefill_tokens":117888,"num_requests":1,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":24451,"score_linear":117888.0,"score_lmetric":142339},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":10347,"ongoing_decode_tokens":0,"pending_prefill_tokens":2667,"num_requests":1,"active_p_offloads":0,"cached_blocks":1531,"cache_hit":0,"new_prefill":24451,"score_linear":10347.0,"score_lmetric":27118},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76969,"ongoing_decode_tokens":0,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1458,"cache_hit":0,"new_prefill":24451,"score_linear":76969.0,"score_lmetric":101420},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1948,"cache_hit":0,"new_prefill":24451,"score_linear":0.0,"score_lmetric":-24451},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1131,"cache_hit":0,"new_prefill":24451,"score_linear":68259.0,"score_lmetric":24451},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1149,"cache_hit":24064,"new_prefill":387,"score_linear":-24064.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1011,"cache_hit":12800,"new_prefill":11651,"score_linear":67446.0,"score_lmetric":11651}],"chosen_score_linear":-24064.0,"chosen_score_lmetric":0,"t_first_token":348746.086777596,"t_first_token_unix":1779779611.1007216,"t_done":348746.677443115,"t_done_unix":1779779611.6913865},{"request_id":"1363440:1:1363440:542","session_id":"1363440","input_length":543,"t_proxy_recv":348747.784343557,"t_decision_unix":1779779612.798285,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":543,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":142090,"ongoing_decode_tokens":0,"pending_prefill_tokens":121610,"num_requests":2,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":543,"score_linear":142090.0,"score_lmetric":244306},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":117888,"ongoing_decode_tokens":0,"pending_prefill_tokens":117888,"num_requests":1,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":543,"score_linear":117888.0,"score_lmetric":118431},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":53289,"ongoing_decode_tokens":0,"pending_prefill_tokens":4137,"num_requests":2,"active_p_offloads":0,"cached_blocks":1536,"cache_hit":0,"new_prefill":543,"score_linear":53289.0,"score_lmetric":9360},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76969,"ongoing_decode_tokens":0,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1458,"cache_hit":0,"new_prefill":543,"score_linear":76969.0,"score_lmetric":77512},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90778,"ongoing_decode_tokens":0,"pending_prefill_tokens":63130,"num_requests":1,"active_p_offloads":0,"cached_blocks":1948,"cache_hit":0,"new_prefill":543,"score_linear":90778.0,"score_lmetric":63673},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1131,"cache_hit":0,"new_prefill":543,"score_linear":68259.0,"score_lmetric":543},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1149,"cache_hit":0,"new_prefill":543,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1011,"cache_hit":0,"new_prefill":543,"score_linear":80246.0,"score_lmetric":543}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348747.859344748,"t_first_token_unix":1779779612.8732882,"t_done":348748.035831271,"t_done_unix":1779779613.0497746},{"request_id":"1355951:2:1363365:540","session_id":"1355951","input_length":37232,"t_proxy_recv":348747.494937594,"t_decision_unix":1779779612.5088792,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":35328,"affinity_cache_ratio":0.9488611946712505,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":35328,"estimated_new_tokens":1904,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":142090,"ongoing_decode_tokens":0,"pending_prefill_tokens":121610,"num_requests":2,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":37232,"score_linear":142090.0,"score_lmetric":317684},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":117888,"ongoing_decode_tokens":0,"pending_prefill_tokens":117888,"num_requests":1,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":37232,"score_linear":117888.0,"score_lmetric":155120},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":16057,"ongoing_decode_tokens":0,"pending_prefill_tokens":2233,"num_requests":1,"active_p_offloads":0,"cached_blocks":1536,"cache_hit":35328,"new_prefill":1904,"score_linear":-19271.0,"score_lmetric":4137},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76969,"ongoing_decode_tokens":0,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1458,"cache_hit":0,"new_prefill":37232,"score_linear":76969.0,"score_lmetric":114201},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":62685,"ongoing_decode_tokens":0,"pending_prefill_tokens":62685,"num_requests":0,"active_p_offloads":0,"cached_blocks":1948,"cache_hit":0,"new_prefill":37232,"score_linear":62685.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1131,"cache_hit":0,"new_prefill":37232,"score_linear":68259.0,"score_lmetric":37232},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1149,"cache_hit":0,"new_prefill":37232,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1011,"cache_hit":0,"new_prefill":37232,"score_linear":80246.0,"score_lmetric":37232}],"chosen_score_linear":-19271.0,"chosen_score_lmetric":4137,"t_first_token":348748.928573789,"t_first_token_unix":1779779613.9425173,"t_done":348749.736818308,"t_done_unix":1779779614.7507615},{"request_id":"1364128:1:1364128:545","session_id":"1364128","input_length":8706,"t_proxy_recv":348749.953118927,"t_decision_unix":1779779614.9670603,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":2050,"tie_break_used":false,"cache_hit":6656,"estimated_new_tokens":2050,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":142090,"ongoing_decode_tokens":0,"pending_prefill_tokens":121610,"num_requests":2,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":8706,"score_linear":142090.0,"score_lmetric":260632},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":117888,"ongoing_decode_tokens":0,"pending_prefill_tokens":117888,"num_requests":1,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":8706,"score_linear":117888.0,"score_lmetric":126594},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":16057,"ongoing_decode_tokens":16057,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1539,"cache_hit":6656,"new_prefill":2050,"score_linear":9401.0,"score_lmetric":2050},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76969,"ongoing_decode_tokens":0,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1458,"cache_hit":0,"new_prefill":8706,"score_linear":76969.0,"score_lmetric":85675},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90778,"ongoing_decode_tokens":0,"pending_prefill_tokens":63130,"num_requests":1,"active_p_offloads":0,"cached_blocks":1948,"cache_hit":0,"new_prefill":8706,"score_linear":90778.0,"score_lmetric":71836},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1131,"cache_hit":0,"new_prefill":8706,"score_linear":68259.0,"score_lmetric":8706},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":12636,"ongoing_decode_tokens":0,"pending_prefill_tokens":8540,"num_requests":1,"active_p_offloads":0,"cached_blocks":1150,"cache_hit":0,"new_prefill":8706,"score_linear":12636.0,"score_lmetric":17246},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":94143,"ongoing_decode_tokens":80246,"pending_prefill_tokens":2633,"num_requests":2,"active_p_offloads":0,"cached_blocks":1011,"cache_hit":0,"new_prefill":8706,"score_linear":94143.0,"score_lmetric":22678}],"chosen_score_linear":9401.0,"chosen_score_lmetric":2050,"t_first_token":348750.174154337,"t_first_token_unix":1779779615.1880975,"t_done":348751.248017358,"t_done_unix":1779779616.2619622},{"request_id":"1364960:1:1364960:549","session_id":"1364960","input_length":2019,"t_proxy_recv":348752.878217319,"t_decision_unix":1779779617.8921587,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.5,"fallback_score":2019,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":2019,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":240128,"ongoing_decode_tokens":0,"pending_prefill_tokens":121856,"num_requests":3,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":2019,"score_linear":240128.0,"score_lmetric":371625},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":117888,"ongoing_decode_tokens":0,"pending_prefill_tokens":117888,"num_requests":1,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":2019,"score_linear":117888.0,"score_lmetric":119907},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":16057,"ongoing_decode_tokens":16057,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1543,"cache_hit":0,"new_prefill":2019,"score_linear":16057.0,"score_lmetric":2019},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76969,"ongoing_decode_tokens":0,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1458,"cache_hit":0,"new_prefill":2019,"score_linear":76969.0,"score_lmetric":78988},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90778,"ongoing_decode_tokens":0,"pending_prefill_tokens":63130,"num_requests":1,"active_p_offloads":0,"cached_blocks":1948,"cache_hit":0,"new_prefill":2019,"score_linear":90778.0,"score_lmetric":65149},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1131,"cache_hit":0,"new_prefill":2019,"score_linear":68259.0,"score_lmetric":2019},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":72150,"ongoing_decode_tokens":12636,"pending_prefill_tokens":35450,"num_requests":2,"active_p_offloads":0,"cached_blocks":1150,"cache_hit":0,"new_prefill":2019,"score_linear":72150.0,"score_lmetric":74938},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":94143,"ongoing_decode_tokens":94143,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1011,"cache_hit":0,"new_prefill":2019,"score_linear":94143.0,"score_lmetric":4038}],"chosen_score_linear":16057.0,"chosen_score_lmetric":2019,"t_first_token":348753.024751976,"t_first_token_unix":1779779618.038695,"t_done":348753.264059461,"t_done_unix":1779779618.2780025},{"request_id":"1305906:2:1363233:539","session_id":"1305906","input_length":16057,"t_proxy_recv":348746.953384038,"t_decision_unix":1779779611.9673223,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":13824,"affinity_cache_ratio":0.8609329264495236,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":13824,"estimated_new_tokens":2233,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":142090,"ongoing_decode_tokens":0,"pending_prefill_tokens":121610,"num_requests":2,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":16057,"score_linear":142090.0,"score_lmetric":275334},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":117888,"ongoing_decode_tokens":0,"pending_prefill_tokens":117888,"num_requests":1,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":16057,"score_linear":117888.0,"score_lmetric":133945},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1536,"cache_hit":13824,"new_prefill":2233,"score_linear":-13824.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76969,"ongoing_decode_tokens":0,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1458,"cache_hit":0,"new_prefill":16057,"score_linear":76969.0,"score_lmetric":93026},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":62685,"ongoing_decode_tokens":0,"pending_prefill_tokens":62685,"num_requests":0,"active_p_offloads":0,"cached_blocks":1948,"cache_hit":0,"new_prefill":16057,"score_linear":62685.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1131,"cache_hit":0,"new_prefill":16057,"score_linear":68259.0,"score_lmetric":16057},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1149,"cache_hit":0,"new_prefill":16057,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1011,"cache_hit":0,"new_prefill":16057,"score_linear":80246.0,"score_lmetric":16057}],"chosen_score_linear":-13824.0,"chosen_score_lmetric":0,"t_first_token":348748.418253471,"t_first_token_unix":1779779613.432197,"t_done":348753.657274633,"t_done_unix":1779779618.6712177},{"request_id":"1363440:2:1365095:551","session_id":"1363440","input_length":3866,"t_proxy_recv":348753.459768654,"t_decision_unix":1779779618.47371,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":6,"chosen_idx":5,"affinity_cache_hit":512,"affinity_cache_ratio":0.13243662700465597,"affinity_num_requests":2,"avg_num_requests":1.625,"fallback_score":3866,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":3866,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":240128,"ongoing_decode_tokens":0,"pending_prefill_tokens":121856,"num_requests":3,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":3866,"score_linear":240128.0,"score_lmetric":377166},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":132609,"ongoing_decode_tokens":0,"pending_prefill_tokens":119297,"num_requests":2,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":3866,"score_linear":132609.0,"score_lmetric":246326},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":16057,"ongoing_decode_tokens":16057,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1547,"cache_hit":0,"new_prefill":3866,"score_linear":16057.0,"score_lmetric":3866},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76969,"ongoing_decode_tokens":0,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1458,"cache_hit":0,"new_prefill":3866,"score_linear":76969.0,"score_lmetric":80835},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90778,"ongoing_decode_tokens":0,"pending_prefill_tokens":63130,"num_requests":1,"active_p_offloads":0,"cached_blocks":1948,"cache_hit":0,"new_prefill":3866,"score_linear":90778.0,"score_lmetric":66996},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1131,"cache_hit":0,"new_prefill":3866,"score_linear":68259.0,"score_lmetric":3866},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":72150,"ongoing_decode_tokens":12636,"pending_prefill_tokens":35450,"num_requests":2,"active_p_offloads":0,"cached_blocks":1150,"cache_hit":512,"new_prefill":3354,"score_linear":71638.0,"score_lmetric":77608},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":94143,"ongoing_decode_tokens":94143,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1011,"cache_hit":0,"new_prefill":3866,"score_linear":94143.0,"score_lmetric":7732}],"chosen_score_linear":68259.0,"chosen_score_lmetric":3866,"t_first_token":348753.787832553,"t_first_token_unix":1779779618.801776,"t_done":348754.287556078,"t_done_unix":1779779619.3014996},{"request_id":"1365440:1:1365440:552","session_id":"1365440","input_length":9913,"t_proxy_recv":348754.684406137,"t_decision_unix":1779779619.6983476,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.5,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":9913,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":240128,"ongoing_decode_tokens":0,"pending_prefill_tokens":121856,"num_requests":3,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":9913,"score_linear":240128.0,"score_lmetric":395307},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":132609,"ongoing_decode_tokens":0,"pending_prefill_tokens":119297,"num_requests":2,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":9913,"score_linear":132609.0,"score_lmetric":258420},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1552,"cache_hit":0,"new_prefill":9913,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76969,"ongoing_decode_tokens":0,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1458,"cache_hit":0,"new_prefill":9913,"score_linear":76969.0,"score_lmetric":86882},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90778,"ongoing_decode_tokens":0,"pending_prefill_tokens":63130,"num_requests":1,"active_p_offloads":0,"cached_blocks":1948,"cache_hit":0,"new_prefill":9913,"score_linear":90778.0,"score_lmetric":73043},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1138,"cache_hit":0,"new_prefill":9913,"score_linear":68259.0,"score_lmetric":9913},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":72150,"ongoing_decode_tokens":12636,"pending_prefill_tokens":35450,"num_requests":2,"active_p_offloads":0,"cached_blocks":1150,"cache_hit":0,"new_prefill":9913,"score_linear":72150.0,"score_lmetric":90726},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":94143,"ongoing_decode_tokens":94143,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1011,"cache_hit":0,"new_prefill":9913,"score_linear":94143.0,"score_lmetric":19826}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348755.442094703,"t_first_token_unix":1779779620.4560382,"t_done":348755.55563915,"t_done_unix":1779779620.5695822},{"request_id":"1364090:1:1364090:544","session_id":"1364090","input_length":13897,"t_proxy_recv":348749.845399294,"t_decision_unix":1779779614.859341,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":2633,"tie_break_used":false,"cache_hit":11264,"estimated_new_tokens":2633,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":142090,"ongoing_decode_tokens":0,"pending_prefill_tokens":121610,"num_requests":2,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":7680,"new_prefill":6217,"score_linear":134410.0,"score_lmetric":255654},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":117888,"ongoing_decode_tokens":0,"pending_prefill_tokens":117888,"num_requests":1,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":13897,"score_linear":117888.0,"score_lmetric":131785},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":16057,"ongoing_decode_tokens":16057,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1539,"cache_hit":0,"new_prefill":13897,"score_linear":16057.0,"score_lmetric":13897},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76969,"ongoing_decode_tokens":0,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1458,"cache_hit":0,"new_prefill":13897,"score_linear":76969.0,"score_lmetric":90866},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90778,"ongoing_decode_tokens":0,"pending_prefill_tokens":63130,"num_requests":1,"active_p_offloads":0,"cached_blocks":1948,"cache_hit":7680,"new_prefill":6217,"score_linear":83098.0,"score_lmetric":69347},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1131,"cache_hit":0,"new_prefill":13897,"score_linear":68259.0,"score_lmetric":13897},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":12636,"ongoing_decode_tokens":0,"pending_prefill_tokens":8540,"num_requests":1,"active_p_offloads":0,"cached_blocks":1150,"cache_hit":7680,"new_prefill":6217,"score_linear":4956.0,"score_lmetric":14757},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1011,"cache_hit":11264,"new_prefill":2633,"score_linear":68982.0,"score_lmetric":2633}],"chosen_score_linear":68982.0,"chosen_score_lmetric":2633,"t_first_token":348750.202210418,"t_first_token_unix":1779779615.2161536,"t_done":348756.568907821,"t_done_unix":1779779621.5828512},{"request_id":"1366128:1:1366128:553","session_id":"1366128","input_length":11154,"t_proxy_recv":348756.973794921,"t_decision_unix":1779779621.987736,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":11154,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":240128,"ongoing_decode_tokens":0,"pending_prefill_tokens":121856,"num_requests":3,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":11154,"score_linear":240128.0,"score_lmetric":399030},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":132609,"ongoing_decode_tokens":0,"pending_prefill_tokens":119297,"num_requests":2,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":11154,"score_linear":132609.0,"score_lmetric":260902},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1571,"cache_hit":0,"new_prefill":11154,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76969,"ongoing_decode_tokens":0,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1458,"cache_hit":0,"new_prefill":11154,"score_linear":76969.0,"score_lmetric":88123},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90778,"ongoing_decode_tokens":0,"pending_prefill_tokens":63130,"num_requests":1,"active_p_offloads":0,"cached_blocks":1948,"cache_hit":0,"new_prefill":11154,"score_linear":90778.0,"score_lmetric":74284},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1138,"cache_hit":0,"new_prefill":11154,"score_linear":68259.0,"score_lmetric":11154},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":72150,"ongoing_decode_tokens":12636,"pending_prefill_tokens":35450,"num_requests":2,"active_p_offloads":0,"cached_blocks":1150,"cache_hit":0,"new_prefill":11154,"score_linear":72150.0,"score_lmetric":93208},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1017,"cache_hit":0,"new_prefill":11154,"score_linear":80246.0,"score_lmetric":11154}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348757.856636368,"t_first_token_unix":1779779622.87058,"t_done":348758.367203264,"t_done_unix":1779779623.381147},{"request_id":"1361694:1:1361694:534","session_id":"1361694","input_length":76969,"t_proxy_recv":348741.725980187,"t_decision_unix":1779779606.7399213,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":76969,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":142090,"ongoing_decode_tokens":0,"pending_prefill_tokens":121610,"num_requests":2,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":76969,"score_linear":142090.0,"score_lmetric":397158},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":117888,"ongoing_decode_tokens":0,"pending_prefill_tokens":117888,"num_requests":1,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":76969,"score_linear":117888.0,"score_lmetric":194857},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":16480,"ongoing_decode_tokens":16480,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1467,"cache_hit":0,"new_prefill":76969,"score_linear":16480.0,"score_lmetric":76969},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1458,"cache_hit":0,"new_prefill":76969,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":109604,"ongoing_decode_tokens":48193,"pending_prefill_tokens":5603,"num_requests":2,"active_p_offloads":0,"cached_blocks":1930,"cache_hit":0,"new_prefill":76969,"score_linear":109604.0,"score_lmetric":165144},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":102938,"ongoing_decode_tokens":102938,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1130,"cache_hit":0,"new_prefill":76969,"score_linear":102938.0,"score_lmetric":153938},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":58758,"ongoing_decode_tokens":58758,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1058,"cache_hit":3584,"new_prefill":73385,"score_linear":55174.0,"score_lmetric":73385},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":101617,"ongoing_decode_tokens":101617,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":995,"cache_hit":0,"new_prefill":76969,"score_linear":101617.0,"score_lmetric":153938}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348761.930566941,"t_first_token_unix":1779779626.9445105,"t_done":348762.994337312,"t_done_unix":1779779628.0082808},{"request_id":"1367312:1:1367312:566","session_id":"1367312","input_length":13749,"t_proxy_recv":348761.306179173,"t_decision_unix":1779779626.3201208,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.25,"fallback_score":13749,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":13749,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":240128,"ongoing_decode_tokens":0,"pending_prefill_tokens":121856,"num_requests":3,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":13749,"score_linear":240128.0,"score_lmetric":406815},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":132609,"ongoing_decode_tokens":0,"pending_prefill_tokens":119297,"num_requests":2,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":13749,"score_linear":132609.0,"score_lmetric":266092},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":43321,"ongoing_decode_tokens":0,"pending_prefill_tokens":23865,"num_requests":3,"active_p_offloads":0,"cached_blocks":1592,"cache_hit":0,"new_prefill":13749,"score_linear":43321.0,"score_lmetric":112842},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76969,"ongoing_decode_tokens":0,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1458,"cache_hit":0,"new_prefill":13749,"score_linear":76969.0,"score_lmetric":90718},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":181278,"ongoing_decode_tokens":90778,"pending_prefill_tokens":8580,"num_requests":3,"active_p_offloads":0,"cached_blocks":1948,"cache_hit":0,"new_prefill":13749,"score_linear":181278.0,"score_lmetric":66987},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":139631,"ongoing_decode_tokens":103330,"pending_prefill_tokens":2509,"num_requests":3,"active_p_offloads":0,"cached_blocks":1138,"cache_hit":0,"new_prefill":13749,"score_linear":139631.0,"score_lmetric":48774},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":72150,"ongoing_decode_tokens":72150,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1150,"cache_hit":0,"new_prefill":13749,"score_linear":72150.0,"score_lmetric":27498},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1017,"cache_hit":0,"new_prefill":13749,"score_linear":80246.0,"score_lmetric":13749}],"chosen_score_linear":80246.0,"chosen_score_lmetric":13749,"t_first_token":348762.495154327,"t_first_token_unix":1779779627.509098,"t_done":348763.13186822,"t_done_unix":1779779628.1458116},{"request_id":"1353821:4:1363399:541","session_id":"1353821","input_length":28093,"t_proxy_recv":348747.620463135,"t_decision_unix":1779779612.6344047,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":27648,"affinity_cache_ratio":0.984159755099135,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":27648,"estimated_new_tokens":445,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":142090,"ongoing_decode_tokens":0,"pending_prefill_tokens":121610,"num_requests":2,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":28093,"score_linear":142090.0,"score_lmetric":299406},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":117888,"ongoing_decode_tokens":0,"pending_prefill_tokens":117888,"num_requests":1,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":28093,"score_linear":117888.0,"score_lmetric":145981},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":53289,"ongoing_decode_tokens":0,"pending_prefill_tokens":4137,"num_requests":2,"active_p_offloads":0,"cached_blocks":1536,"cache_hit":0,"new_prefill":28093,"score_linear":53289.0,"score_lmetric":64460},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76969,"ongoing_decode_tokens":0,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1458,"cache_hit":0,"new_prefill":28093,"score_linear":76969.0,"score_lmetric":105062},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":62685,"ongoing_decode_tokens":0,"pending_prefill_tokens":62685,"num_requests":0,"active_p_offloads":0,"cached_blocks":1948,"cache_hit":27648,"new_prefill":445,"score_linear":35037.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1131,"cache_hit":0,"new_prefill":28093,"score_linear":68259.0,"score_lmetric":28093},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1149,"cache_hit":0,"new_prefill":28093,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1011,"cache_hit":0,"new_prefill":28093,"score_linear":80246.0,"score_lmetric":28093}],"chosen_score_linear":35037.0,"chosen_score_lmetric":0,"t_first_token":348760.539701335,"t_first_token_unix":1779779625.5536442,"t_done":348763.184257633,"t_done_unix":1779779628.198201},{"request_id":"1362265:2:1367373:568","session_id":"1362265","input_length":10134,"t_proxy_recv":348761.511150984,"t_decision_unix":1779779626.5250924,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":8704,"affinity_cache_ratio":0.8588908624432603,"affinity_num_requests":3,"avg_num_requests":2.5,"fallback_score":null,"tie_break_used":false,"cache_hit":8704,"estimated_new_tokens":1430,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":240128,"ongoing_decode_tokens":0,"pending_prefill_tokens":121856,"num_requests":3,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":10134,"score_linear":240128.0,"score_lmetric":395970},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":132609,"ongoing_decode_tokens":0,"pending_prefill_tokens":119297,"num_requests":2,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":10134,"score_linear":132609.0,"score_lmetric":258862},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":43321,"ongoing_decode_tokens":0,"pending_prefill_tokens":23865,"num_requests":3,"active_p_offloads":0,"cached_blocks":1592,"cache_hit":8704,"new_prefill":1430,"score_linear":34617.0,"score_lmetric":75885},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76969,"ongoing_decode_tokens":0,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1458,"cache_hit":0,"new_prefill":10134,"score_linear":76969.0,"score_lmetric":87103},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":181278,"ongoing_decode_tokens":90778,"pending_prefill_tokens":8580,"num_requests":3,"active_p_offloads":0,"cached_blocks":1948,"cache_hit":0,"new_prefill":10134,"score_linear":181278.0,"score_lmetric":56142},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":139631,"ongoing_decode_tokens":103330,"pending_prefill_tokens":2509,"num_requests":3,"active_p_offloads":0,"cached_blocks":1138,"cache_hit":0,"new_prefill":10134,"score_linear":139631.0,"score_lmetric":37929},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":98987,"ongoing_decode_tokens":72150,"pending_prefill_tokens":26837,"num_requests":3,"active_p_offloads":0,"cached_blocks":1150,"cache_hit":0,"new_prefill":10134,"score_linear":98987.0,"score_lmetric":110913},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":93995,"ongoing_decode_tokens":80246,"pending_prefill_tokens":13749,"num_requests":2,"active_p_offloads":0,"cached_blocks":1017,"cache_hit":0,"new_prefill":10134,"score_linear":93995.0,"score_lmetric":47766}],"chosen_score_linear":34617.0,"chosen_score_lmetric":75885,"t_first_token":348763.341143015,"t_first_token_unix":1779779628.3550863,"t_done":348763.702769858,"t_done_unix":1779779628.7167137},{"request_id":"1363093:1:1363093:538","session_id":"1363093","input_length":62685,"t_proxy_recv":348746.469363811,"t_decision_unix":1779779611.4833052,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":-62685,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":62685,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":142090,"ongoing_decode_tokens":0,"pending_prefill_tokens":121610,"num_requests":2,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":62685,"score_linear":142090.0,"score_lmetric":368590},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":117888,"ongoing_decode_tokens":0,"pending_prefill_tokens":117888,"num_requests":1,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":62685,"score_linear":117888.0,"score_lmetric":180573},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1536,"cache_hit":0,"new_prefill":62685,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76969,"ongoing_decode_tokens":0,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1458,"cache_hit":0,"new_prefill":62685,"score_linear":76969.0,"score_lmetric":139654},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":1948,"cache_hit":0,"new_prefill":62685,"score_linear":0.0,"score_lmetric":-62685},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1131,"cache_hit":0,"new_prefill":62685,"score_linear":68259.0,"score_lmetric":62685},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":24451,"ongoing_decode_tokens":24451,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1149,"cache_hit":0,"new_prefill":62685,"score_linear":24451.0,"score_lmetric":62685},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1011,"cache_hit":0,"new_prefill":62685,"score_linear":80246.0,"score_lmetric":62685}],"chosen_score_linear":0.0,"chosen_score_lmetric":-62685,"t_first_token":348760.538184105,"t_first_token_unix":1779779625.5521278,"t_done":348763.785887886,"t_done_unix":1779779628.7998316},{"request_id":"1364128:2:1367249:565","session_id":"1364128","input_length":9095,"t_proxy_recv":348761.104511628,"t_decision_unix":1779779626.118453,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":8704,"affinity_cache_ratio":0.9570093457943926,"affinity_num_requests":2,"avg_num_requests":2.125,"fallback_score":null,"tie_break_used":false,"cache_hit":8704,"estimated_new_tokens":391,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":240128,"ongoing_decode_tokens":0,"pending_prefill_tokens":121856,"num_requests":3,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":9095,"score_linear":240128.0,"score_lmetric":392853},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":132609,"ongoing_decode_tokens":0,"pending_prefill_tokens":119297,"num_requests":2,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":9095,"score_linear":132609.0,"score_lmetric":256784},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":34226,"ongoing_decode_tokens":0,"pending_prefill_tokens":23474,"num_requests":2,"active_p_offloads":0,"cached_blocks":1592,"cache_hit":8704,"new_prefill":391,"score_linear":25522.0,"score_lmetric":47730},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76969,"ongoing_decode_tokens":0,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1458,"cache_hit":0,"new_prefill":9095,"score_linear":76969.0,"score_lmetric":86064},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":181278,"ongoing_decode_tokens":90778,"pending_prefill_tokens":8580,"num_requests":3,"active_p_offloads":0,"cached_blocks":1948,"cache_hit":0,"new_prefill":9095,"score_linear":181278.0,"score_lmetric":53025},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":139631,"ongoing_decode_tokens":103330,"pending_prefill_tokens":2509,"num_requests":3,"active_p_offloads":0,"cached_blocks":1138,"cache_hit":0,"new_prefill":9095,"score_linear":139631.0,"score_lmetric":34812},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":72150,"ongoing_decode_tokens":72150,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1150,"cache_hit":0,"new_prefill":9095,"score_linear":72150.0,"score_lmetric":18190},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1017,"cache_hit":0,"new_prefill":9095,"score_linear":80246.0,"score_lmetric":9095}],"chosen_score_linear":25522.0,"chosen_score_lmetric":47730,"t_first_token":348763.266569225,"t_first_token_unix":1779779628.2805123,"t_done":348763.94112677,"t_done_unix":1779779628.95507},{"request_id":"1367424:1:1367424:569","session_id":"1367424","input_length":3392,"t_proxy_recv":348761.710542455,"t_decision_unix":1779779626.7244842,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.625,"fallback_score":17703,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":3392,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":240128,"ongoing_decode_tokens":0,"pending_prefill_tokens":121856,"num_requests":3,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":3392,"score_linear":240128.0,"score_lmetric":375744},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":132609,"ongoing_decode_tokens":0,"pending_prefill_tokens":119297,"num_requests":2,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":3392,"score_linear":132609.0,"score_lmetric":245378},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":53455,"ongoing_decode_tokens":0,"pending_prefill_tokens":25295,"num_requests":4,"active_p_offloads":0,"cached_blocks":1592,"cache_hit":0,"new_prefill":3392,"score_linear":53455.0,"score_lmetric":114748},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76969,"ongoing_decode_tokens":0,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1458,"cache_hit":0,"new_prefill":3392,"score_linear":76969.0,"score_lmetric":80361},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":181278,"ongoing_decode_tokens":90778,"pending_prefill_tokens":8580,"num_requests":3,"active_p_offloads":0,"cached_blocks":1948,"cache_hit":0,"new_prefill":3392,"score_linear":181278.0,"score_lmetric":35916},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":139631,"ongoing_decode_tokens":103330,"pending_prefill_tokens":2509,"num_requests":3,"active_p_offloads":0,"cached_blocks":1138,"cache_hit":0,"new_prefill":3392,"score_linear":139631.0,"score_lmetric":17703},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":98987,"ongoing_decode_tokens":72150,"pending_prefill_tokens":26837,"num_requests":3,"active_p_offloads":0,"cached_blocks":1150,"cache_hit":0,"new_prefill":3392,"score_linear":98987.0,"score_lmetric":90687},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":93995,"ongoing_decode_tokens":80246,"pending_prefill_tokens":13749,"num_requests":2,"active_p_offloads":0,"cached_blocks":1017,"cache_hit":0,"new_prefill":3392,"score_linear":93995.0,"score_lmetric":34282}],"chosen_score_linear":139631.0,"chosen_score_lmetric":17703,"t_first_token":348763.774400166,"t_first_token_unix":1779779628.7883437,"t_done":348763.951612307,"t_done_unix":1779779628.9655554},{"request_id":"1344773:11:1366387:555","session_id":"1344773","input_length":35071,"t_proxy_recv":348757.947517971,"t_decision_unix":1779779622.9614594,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":34816,"affinity_cache_ratio":0.9927290353853612,"affinity_num_requests":1,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":34816,"estimated_new_tokens":255,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":240128,"ongoing_decode_tokens":0,"pending_prefill_tokens":121856,"num_requests":3,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":35071,"score_linear":240128.0,"score_lmetric":470781},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":132609,"ongoing_decode_tokens":0,"pending_prefill_tokens":119297,"num_requests":2,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":35071,"score_linear":132609.0,"score_lmetric":308736},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":11154,"ongoing_decode_tokens":11154,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1571,"cache_hit":0,"new_prefill":35071,"score_linear":11154.0,"score_lmetric":35071},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76969,"ongoing_decode_tokens":0,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1458,"cache_hit":0,"new_prefill":35071,"score_linear":76969.0,"score_lmetric":112040},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90778,"ongoing_decode_tokens":0,"pending_prefill_tokens":63130,"num_requests":1,"active_p_offloads":0,"cached_blocks":1948,"cache_hit":0,"new_prefill":35071,"score_linear":90778.0,"score_lmetric":98201},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1138,"cache_hit":34816,"new_prefill":255,"score_linear":33443.0,"score_lmetric":255},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":72150,"ongoing_decode_tokens":12636,"pending_prefill_tokens":35450,"num_requests":2,"active_p_offloads":0,"cached_blocks":1150,"cache_hit":0,"new_prefill":35071,"score_linear":72150.0,"score_lmetric":141042},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1017,"cache_hit":0,"new_prefill":35071,"score_linear":80246.0,"score_lmetric":35071}],"chosen_score_linear":33443.0,"chosen_score_lmetric":255,"t_first_token":348758.055531439,"t_first_token_unix":1779779623.0694747,"t_done":348763.996967904,"t_done_unix":1779779629.010911},{"request_id":"1366128:2:1367220:563","session_id":"1366128","input_length":15411,"t_proxy_recv":348761.008947487,"t_decision_unix":1779779626.022889,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":10752,"affinity_cache_ratio":0.6976834728440724,"affinity_num_requests":1,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"cache_hit":10752,"estimated_new_tokens":4659,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":240128,"ongoing_decode_tokens":0,"pending_prefill_tokens":121856,"num_requests":3,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":15411,"score_linear":240128.0,"score_lmetric":411801},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":132609,"ongoing_decode_tokens":0,"pending_prefill_tokens":119297,"num_requests":2,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":15411,"score_linear":132609.0,"score_lmetric":269416},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":18815,"ongoing_decode_tokens":0,"pending_prefill_tokens":18815,"num_requests":1,"active_p_offloads":0,"cached_blocks":1592,"cache_hit":10752,"new_prefill":4659,"score_linear":8063.0,"score_lmetric":23474},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76969,"ongoing_decode_tokens":0,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1458,"cache_hit":0,"new_prefill":15411,"score_linear":76969.0,"score_lmetric":92380},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":131795,"ongoing_decode_tokens":90778,"pending_prefill_tokens":7737,"num_requests":2,"active_p_offloads":0,"cached_blocks":1948,"cache_hit":0,"new_prefill":15411,"score_linear":131795.0,"score_lmetric":46296},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":139631,"ongoing_decode_tokens":103330,"pending_prefill_tokens":2509,"num_requests":3,"active_p_offloads":0,"cached_blocks":1138,"cache_hit":0,"new_prefill":15411,"score_linear":139631.0,"score_lmetric":53760},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":72150,"ongoing_decode_tokens":72150,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1150,"cache_hit":0,"new_prefill":15411,"score_linear":72150.0,"score_lmetric":30822},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1017,"cache_hit":0,"new_prefill":15411,"score_linear":80246.0,"score_lmetric":15411}],"chosen_score_linear":8063.0,"chosen_score_lmetric":23474,"t_first_token":348763.26586598,"t_first_token_unix":1779779628.279809,"t_done":348764.138381451,"t_done_unix":1779779629.1523252},{"request_id":"1367312:2:1368064:574","session_id":"1367312","input_length":15501,"t_proxy_recv":348764.005768173,"t_decision_unix":1779779629.0197096,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":13312,"affinity_cache_ratio":0.8587833043029482,"affinity_num_requests":1,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"cache_hit":13312,"estimated_new_tokens":2189,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":240128,"ongoing_decode_tokens":0,"pending_prefill_tokens":121856,"num_requests":3,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":15501,"score_linear":240128.0,"score_lmetric":412071},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":132609,"ongoing_decode_tokens":0,"pending_prefill_tokens":119297,"num_requests":2,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":15501,"score_linear":132609.0,"score_lmetric":269596},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":34226,"ongoing_decode_tokens":34226,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1594,"cache_hit":0,"new_prefill":15501,"score_linear":34226.0,"score_lmetric":31002},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":72919,"ongoing_decode_tokens":0,"pending_prefill_tokens":72919,"num_requests":1,"active_p_offloads":0,"cached_blocks":1608,"cache_hit":0,"new_prefill":15501,"score_linear":72919.0,"score_lmetric":88420},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90500,"ongoing_decode_tokens":90500,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2070,"cache_hit":0,"new_prefill":15501,"score_linear":90500.0,"score_lmetric":15501},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":104560,"ongoing_decode_tokens":104560,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":15501,"score_linear":104560.0,"score_lmetric":31002},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":98987,"ongoing_decode_tokens":72150,"pending_prefill_tokens":26837,"num_requests":3,"active_p_offloads":0,"cached_blocks":1150,"cache_hit":0,"new_prefill":15501,"score_linear":98987.0,"score_lmetric":127014},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1043,"cache_hit":13312,"new_prefill":2189,"score_linear":66934.0,"score_lmetric":2189}],"chosen_score_linear":66934.0,"chosen_score_lmetric":2189,"t_first_token":348764.328774709,"t_first_token_unix":1779779629.3427181,"t_done":348764.9593873,"t_done_unix":1779779629.9733305},{"request_id":"1364960:2:1368239:578","session_id":"1364960","input_length":7048,"t_proxy_recv":348764.605459555,"t_decision_unix":1779779629.6194007,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":2048,"affinity_cache_ratio":0.2905788876276958,"affinity_num_requests":2,"avg_num_requests":2.25,"fallback_score":10000,"tie_break_used":false,"cache_hit":2048,"estimated_new_tokens":5000,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":240128,"ongoing_decode_tokens":0,"pending_prefill_tokens":121856,"num_requests":3,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":7048,"score_linear":240128.0,"score_lmetric":386712},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":132609,"ongoing_decode_tokens":0,"pending_prefill_tokens":119297,"num_requests":2,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":7048,"score_linear":132609.0,"score_lmetric":252690},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":27660,"ongoing_decode_tokens":27660,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1603,"cache_hit":2048,"new_prefill":5000,"score_linear":25612.0,"score_lmetric":10000},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":72919,"ongoing_decode_tokens":0,"pending_prefill_tokens":72919,"num_requests":1,"active_p_offloads":0,"cached_blocks":1608,"cache_hit":0,"new_prefill":7048,"score_linear":72919.0,"score_lmetric":79967},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":112273,"ongoing_decode_tokens":90500,"pending_prefill_tokens":21773,"num_requests":2,"active_p_offloads":0,"cached_blocks":2070,"cache_hit":0,"new_prefill":7048,"score_linear":112273.0,"score_lmetric":57642},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":139754,"ongoing_decode_tokens":139754,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":7048,"score_linear":139754.0,"score_lmetric":21144},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":98987,"ongoing_decode_tokens":72150,"pending_prefill_tokens":26837,"num_requests":3,"active_p_offloads":0,"cached_blocks":1150,"cache_hit":0,"new_prefill":7048,"score_linear":98987.0,"score_lmetric":101655},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":95747,"ongoing_decode_tokens":95747,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1043,"cache_hit":0,"new_prefill":7048,"score_linear":95747.0,"score_lmetric":14096}],"chosen_score_linear":25612.0,"chosen_score_lmetric":10000,"t_first_token":348764.991507819,"t_first_token_unix":1779779630.0054512,"t_done":348765.468285689,"t_done_unix":1779779630.482229},{"request_id":"1277428:5:1366630:559","session_id":"1277428","input_length":36301,"t_proxy_recv":348758.774193099,"t_decision_unix":1779779623.7881346,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":33792,"affinity_cache_ratio":0.9308834467370045,"affinity_num_requests":2,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"cache_hit":33792,"estimated_new_tokens":2509,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":240128,"ongoing_decode_tokens":0,"pending_prefill_tokens":121856,"num_requests":3,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":36301,"score_linear":240128.0,"score_lmetric":474471},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":132609,"ongoing_decode_tokens":0,"pending_prefill_tokens":119297,"num_requests":2,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":4608,"new_prefill":31693,"score_linear":128001.0,"score_lmetric":301980},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1592,"cache_hit":0,"new_prefill":36301,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76969,"ongoing_decode_tokens":0,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1458,"cache_hit":17408,"new_prefill":18893,"score_linear":59561.0,"score_lmetric":95862},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":131795,"ongoing_decode_tokens":0,"pending_prefill_tokens":70867,"num_requests":2,"active_p_offloads":0,"cached_blocks":1948,"cache_hit":0,"new_prefill":36301,"score_linear":131795.0,"score_lmetric":214336},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":103330,"ongoing_decode_tokens":103330,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1138,"cache_hit":33792,"new_prefill":2509,"score_linear":69538.0,"score_lmetric":5018},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":72150,"ongoing_decode_tokens":12636,"pending_prefill_tokens":35450,"num_requests":2,"active_p_offloads":0,"cached_blocks":1150,"cache_hit":0,"new_prefill":36301,"score_linear":72150.0,"score_lmetric":143502},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1017,"cache_hit":0,"new_prefill":36301,"score_linear":80246.0,"score_lmetric":36301}],"chosen_score_linear":69538.0,"chosen_score_lmetric":5018,"t_first_token":348763.511558018,"t_first_token_unix":1779779628.5255013,"t_done":348765.637787663,"t_done_unix":1779779630.6517308},{"request_id":"1367329:1:1367329:567","session_id":"1367329","input_length":26837,"t_proxy_recv":348761.366621894,"t_decision_unix":1779779626.3805633,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.375,"fallback_score":53674,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":26837,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":240128,"ongoing_decode_tokens":0,"pending_prefill_tokens":121856,"num_requests":3,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":26837,"score_linear":240128.0,"score_lmetric":446079},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":132609,"ongoing_decode_tokens":0,"pending_prefill_tokens":119297,"num_requests":2,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":26837,"score_linear":132609.0,"score_lmetric":292268},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":43321,"ongoing_decode_tokens":0,"pending_prefill_tokens":23865,"num_requests":3,"active_p_offloads":0,"cached_blocks":1592,"cache_hit":0,"new_prefill":26837,"score_linear":43321.0,"score_lmetric":152106},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76969,"ongoing_decode_tokens":0,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1458,"cache_hit":0,"new_prefill":26837,"score_linear":76969.0,"score_lmetric":103806},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":181278,"ongoing_decode_tokens":90778,"pending_prefill_tokens":8580,"num_requests":3,"active_p_offloads":0,"cached_blocks":1948,"cache_hit":0,"new_prefill":26837,"score_linear":181278.0,"score_lmetric":106251},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":139631,"ongoing_decode_tokens":103330,"pending_prefill_tokens":2509,"num_requests":3,"active_p_offloads":0,"cached_blocks":1138,"cache_hit":0,"new_prefill":26837,"score_linear":139631.0,"score_lmetric":88038},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":72150,"ongoing_decode_tokens":72150,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1150,"cache_hit":0,"new_prefill":26837,"score_linear":72150.0,"score_lmetric":53674},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":93995,"ongoing_decode_tokens":80246,"pending_prefill_tokens":13749,"num_requests":2,"active_p_offloads":0,"cached_blocks":1017,"cache_hit":0,"new_prefill":26837,"score_linear":93995.0,"score_lmetric":81172}],"chosen_score_linear":72150.0,"chosen_score_lmetric":53674,"t_first_token":348764.699405594,"t_first_token_unix":1779779629.713349,"t_done":348765.884105083,"t_done_unix":1779779630.8980482},{"request_id":"1367169:1:1367169:562","session_id":"1367169","input_length":18815,"t_proxy_recv":348760.821892014,"t_decision_unix":1779779625.8358316,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.75,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":18815,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":240128,"ongoing_decode_tokens":0,"pending_prefill_tokens":121856,"num_requests":3,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":18815,"score_linear":240128.0,"score_lmetric":422013},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":132609,"ongoing_decode_tokens":0,"pending_prefill_tokens":119297,"num_requests":2,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":18815,"score_linear":132609.0,"score_lmetric":276224},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1592,"cache_hit":0,"new_prefill":18815,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76969,"ongoing_decode_tokens":0,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1458,"cache_hit":0,"new_prefill":18815,"score_linear":76969.0,"score_lmetric":95784},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":131795,"ongoing_decode_tokens":90778,"pending_prefill_tokens":7737,"num_requests":2,"active_p_offloads":0,"cached_blocks":1948,"cache_hit":0,"new_prefill":18815,"score_linear":131795.0,"score_lmetric":53104},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":139631,"ongoing_decode_tokens":103330,"pending_prefill_tokens":2509,"num_requests":3,"active_p_offloads":0,"cached_blocks":1138,"cache_hit":0,"new_prefill":18815,"score_linear":139631.0,"score_lmetric":63972},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":72150,"ongoing_decode_tokens":72150,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1150,"cache_hit":11776,"new_prefill":7039,"score_linear":60374.0,"score_lmetric":14078},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1017,"cache_hit":0,"new_prefill":18815,"score_linear":80246.0,"score_lmetric":18815}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348763.264112753,"t_first_token_unix":1779779628.2780561,"t_done":348765.939559787,"t_done_unix":1779779630.9535027},{"request_id":"1363943:1:1363943:543","session_id":"1363943","input_length":12636,"t_proxy_recv":348749.467593368,"t_decision_unix":1779779614.4815347,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":false,"cache_hit":4096,"estimated_new_tokens":8540,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":142090,"ongoing_decode_tokens":0,"pending_prefill_tokens":121610,"num_requests":2,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":8704,"new_prefill":3932,"score_linear":133386.0,"score_lmetric":251084},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":117888,"ongoing_decode_tokens":0,"pending_prefill_tokens":117888,"num_requests":1,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":12636,"score_linear":117888.0,"score_lmetric":130524},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":53289,"ongoing_decode_tokens":53289,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1536,"cache_hit":0,"new_prefill":12636,"score_linear":53289.0,"score_lmetric":25272},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76969,"ongoing_decode_tokens":0,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1458,"cache_hit":0,"new_prefill":12636,"score_linear":76969.0,"score_lmetric":89605},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90778,"ongoing_decode_tokens":0,"pending_prefill_tokens":63130,"num_requests":1,"active_p_offloads":0,"cached_blocks":1948,"cache_hit":4096,"new_prefill":8540,"score_linear":86682.0,"score_lmetric":71670},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1131,"cache_hit":0,"new_prefill":12636,"score_linear":68259.0,"score_lmetric":12636},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1150,"cache_hit":4096,"new_prefill":8540,"score_linear":-4096.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1011,"cache_hit":4096,"new_prefill":8540,"score_linear":76150.0,"score_lmetric":8540}],"chosen_score_linear":-4096.0,"chosen_score_lmetric":0,"t_first_token":348750.31184486,"t_first_token_unix":1779779615.3257887,"t_done":348765.953452849,"t_done_unix":1779779630.9673958},{"request_id":"1314357:4:1352612:497","session_id":"1314357","input_length":68259,"t_proxy_recv":348709.624854159,"t_decision_unix":1779779574.6387959,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":67072,"affinity_cache_ratio":0.9826103517484874,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":67072,"estimated_new_tokens":1187,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":96026,"ongoing_decode_tokens":96026,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1129,"cache_hit":0,"new_prefill":68259,"score_linear":96026.0,"score_lmetric":68259},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":968,"cache_hit":0,"new_prefill":68259,"score_linear":62618.0,"score_lmetric":68259},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":18840,"ongoing_decode_tokens":0,"pending_prefill_tokens":18840,"num_requests":1,"active_p_offloads":0,"cached_blocks":1362,"cache_hit":0,"new_prefill":68259,"score_linear":18840.0,"score_lmetric":87099},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":10575,"ongoing_decode_tokens":10575,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1270,"cache_hit":0,"new_prefill":68259,"score_linear":10575.0,"score_lmetric":68259},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":76566,"ongoing_decode_tokens":55130,"pending_prefill_tokens":9148,"num_requests":1,"active_p_offloads":0,"cached_blocks":1813,"cache_hit":56832,"new_prefill":11427,"score_linear":19734.0,"score_lmetric":20575},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1125,"cache_hit":67072,"new_prefill":1187,"score_linear":-67072.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1027,"cache_hit":0,"new_prefill":68259,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":992,"cache_hit":0,"new_prefill":68259,"score_linear":21371.0,"score_lmetric":68259}],"chosen_score_linear":-67072.0,"chosen_score_lmetric":0,"t_first_token":348710.122777245,"t_first_token_unix":1779779575.1367207,"t_done":348766.594653172,"t_done_unix":1779779631.6085966},{"request_id":"1344773:12:1367864:571","session_id":"1344773","input_length":35194,"t_proxy_recv":348764.009736156,"t_decision_unix":1779779629.023678,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":34816,"affinity_cache_ratio":0.9892595328749219,"affinity_num_requests":2,"avg_num_requests":2.0,"fallback_score":null,"tie_break_used":false,"cache_hit":34816,"estimated_new_tokens":378,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":240128,"ongoing_decode_tokens":0,"pending_prefill_tokens":121856,"num_requests":3,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":35194,"score_linear":240128.0,"score_lmetric":471150},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":132609,"ongoing_decode_tokens":0,"pending_prefill_tokens":119297,"num_requests":2,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":35194,"score_linear":132609.0,"score_lmetric":308982},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":34226,"ongoing_decode_tokens":34226,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1594,"cache_hit":0,"new_prefill":35194,"score_linear":34226.0,"score_lmetric":70388},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":72919,"ongoing_decode_tokens":0,"pending_prefill_tokens":72919,"num_requests":1,"active_p_offloads":0,"cached_blocks":1608,"cache_hit":0,"new_prefill":35194,"score_linear":72919.0,"score_lmetric":108113},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90500,"ongoing_decode_tokens":90500,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2070,"cache_hit":0,"new_prefill":35194,"score_linear":90500.0,"score_lmetric":35194},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":104560,"ongoing_decode_tokens":104560,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":34816,"new_prefill":378,"score_linear":69744.0,"score_lmetric":756},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":98987,"ongoing_decode_tokens":72150,"pending_prefill_tokens":26837,"num_requests":3,"active_p_offloads":0,"cached_blocks":1150,"cache_hit":0,"new_prefill":35194,"score_linear":98987.0,"score_lmetric":186093},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":95747,"ongoing_decode_tokens":80246,"pending_prefill_tokens":2189,"num_requests":2,"active_p_offloads":0,"cached_blocks":1043,"cache_hit":0,"new_prefill":35194,"score_linear":95747.0,"score_lmetric":74766}],"chosen_score_linear":69744.0,"chosen_score_lmetric":756,"t_first_token":348764.102025723,"t_first_token_unix":1779779629.1159694,"t_done":348766.61168125,"t_done_unix":1779779631.6256247},{"request_id":"1368132:1:1368132:577","session_id":"1368132","input_length":8845,"t_proxy_recv":348764.273190144,"t_decision_unix":1779779629.2871313,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.125,"fallback_score":2189,"tie_break_used":false,"cache_hit":6656,"estimated_new_tokens":2189,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":240128,"ongoing_decode_tokens":0,"pending_prefill_tokens":121856,"num_requests":3,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":8845,"score_linear":240128.0,"score_lmetric":392103},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":132609,"ongoing_decode_tokens":0,"pending_prefill_tokens":119297,"num_requests":2,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":8845,"score_linear":132609.0,"score_lmetric":256284},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":18815,"ongoing_decode_tokens":18815,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1603,"cache_hit":6656,"new_prefill":2189,"score_linear":12159.0,"score_lmetric":2189},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":72919,"ongoing_decode_tokens":0,"pending_prefill_tokens":72919,"num_requests":1,"active_p_offloads":0,"cached_blocks":1608,"cache_hit":0,"new_prefill":8845,"score_linear":72919.0,"score_lmetric":81764},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":112273,"ongoing_decode_tokens":90500,"pending_prefill_tokens":21773,"num_requests":2,"active_p_offloads":0,"cached_blocks":2070,"cache_hit":0,"new_prefill":8845,"score_linear":112273.0,"score_lmetric":61236},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":139754,"ongoing_decode_tokens":139754,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":8845,"score_linear":139754.0,"score_lmetric":26535},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":98987,"ongoing_decode_tokens":72150,"pending_prefill_tokens":26837,"num_requests":3,"active_p_offloads":0,"cached_blocks":1150,"cache_hit":0,"new_prefill":8845,"score_linear":98987.0,"score_lmetric":107046},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":95747,"ongoing_decode_tokens":80246,"pending_prefill_tokens":2189,"num_requests":2,"active_p_offloads":0,"cached_blocks":1043,"cache_hit":0,"new_prefill":8845,"score_linear":95747.0,"score_lmetric":22068}],"chosen_score_linear":12159.0,"chosen_score_lmetric":2189,"t_first_token":348764.511816825,"t_first_token_unix":1779779629.5257607,"t_done":348767.012339683,"t_done_unix":1779779632.0262847},{"request_id":"1368070:1:1368070:575","session_id":"1368070","input_length":21773,"t_proxy_recv":348764.026656027,"t_decision_unix":1779779629.0405977,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.125,"fallback_score":21773,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":21773,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":240128,"ongoing_decode_tokens":0,"pending_prefill_tokens":121856,"num_requests":3,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":21773,"score_linear":240128.0,"score_lmetric":430887},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":132609,"ongoing_decode_tokens":0,"pending_prefill_tokens":119297,"num_requests":2,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":21773,"score_linear":132609.0,"score_lmetric":282140},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":34226,"ongoing_decode_tokens":34226,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1594,"cache_hit":0,"new_prefill":21773,"score_linear":34226.0,"score_lmetric":43546},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":72919,"ongoing_decode_tokens":0,"pending_prefill_tokens":72919,"num_requests":1,"active_p_offloads":0,"cached_blocks":1608,"cache_hit":0,"new_prefill":21773,"score_linear":72919.0,"score_lmetric":94692},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90500,"ongoing_decode_tokens":90500,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2070,"cache_hit":0,"new_prefill":21773,"score_linear":90500.0,"score_lmetric":21773},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":139754,"ongoing_decode_tokens":104560,"pending_prefill_tokens":378,"num_requests":3,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":21773,"score_linear":139754.0,"score_lmetric":66453},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":98987,"ongoing_decode_tokens":72150,"pending_prefill_tokens":26837,"num_requests":3,"active_p_offloads":0,"cached_blocks":1150,"cache_hit":0,"new_prefill":21773,"score_linear":98987.0,"score_lmetric":145830},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":95747,"ongoing_decode_tokens":80246,"pending_prefill_tokens":2189,"num_requests":2,"active_p_offloads":0,"cached_blocks":1043,"cache_hit":0,"new_prefill":21773,"score_linear":95747.0,"score_lmetric":47924}],"chosen_score_linear":90500.0,"chosen_score_lmetric":21773,"t_first_token":348766.417490439,"t_first_token_unix":1779779631.4314427,"t_done":348767.204240751,"t_done_unix":1779779632.2181842},{"request_id":"1270340:4:1366608:558","session_id":"1270340","input_length":41017,"t_proxy_recv":348758.704193791,"t_decision_unix":1779779623.7181354,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":33280,"affinity_cache_ratio":0.8113708949947582,"affinity_num_requests":1,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":33280,"estimated_new_tokens":7737,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":240128,"ongoing_decode_tokens":0,"pending_prefill_tokens":121856,"num_requests":3,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":41017,"score_linear":240128.0,"score_lmetric":488619},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":132609,"ongoing_decode_tokens":0,"pending_prefill_tokens":119297,"num_requests":2,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":41017,"score_linear":132609.0,"score_lmetric":320628},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1592,"cache_hit":0,"new_prefill":41017,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76969,"ongoing_decode_tokens":0,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1458,"cache_hit":1536,"new_prefill":39481,"score_linear":75433.0,"score_lmetric":116450},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90778,"ongoing_decode_tokens":0,"pending_prefill_tokens":63130,"num_requests":1,"active_p_offloads":0,"cached_blocks":1948,"cache_hit":33280,"new_prefill":7737,"score_linear":57498.0,"score_lmetric":70867},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":103330,"ongoing_decode_tokens":103330,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1138,"cache_hit":0,"new_prefill":41017,"score_linear":103330.0,"score_lmetric":82034},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":72150,"ongoing_decode_tokens":12636,"pending_prefill_tokens":35450,"num_requests":2,"active_p_offloads":0,"cached_blocks":1150,"cache_hit":0,"new_prefill":41017,"score_linear":72150.0,"score_lmetric":152934},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1017,"cache_hit":0,"new_prefill":41017,"score_linear":80246.0,"score_lmetric":41017}],"chosen_score_linear":57498.0,"chosen_score_lmetric":70867,"t_first_token":348762.426483375,"t_first_token_unix":1779779627.4404275,"t_done":348767.832381421,"t_done_unix":1779779632.8463273},{"request_id":"1369172:1:1369172:581","session_id":"1369172","input_length":6345,"t_proxy_recv":348767.870320876,"t_decision_unix":1779779632.8842623,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":6345,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":240128,"ongoing_decode_tokens":0,"pending_prefill_tokens":121856,"num_requests":3,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":6345,"score_linear":240128.0,"score_lmetric":384603},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":132609,"ongoing_decode_tokens":0,"pending_prefill_tokens":119297,"num_requests":2,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":6345,"score_linear":132609.0,"score_lmetric":251284},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1653,"cache_hit":0,"new_prefill":6345,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":72919,"ongoing_decode_tokens":0,"pending_prefill_tokens":72919,"num_requests":1,"active_p_offloads":0,"cached_blocks":1608,"cache_hit":0,"new_prefill":6345,"score_linear":72919.0,"score_lmetric":79264},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":112718,"ongoing_decode_tokens":112718,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2127,"cache_hit":0,"new_prefill":6345,"score_linear":112718.0,"score_lmetric":6345},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1154,"cache_hit":0,"new_prefill":6345,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":72498,"ongoing_decode_tokens":72498,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1219,"cache_hit":0,"new_prefill":6345,"score_linear":72498.0,"score_lmetric":12690},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1047,"cache_hit":0,"new_prefill":6345,"score_linear":80246.0,"score_lmetric":6345}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348768.287937471,"t_first_token_unix":1779779633.301881,"t_done":348768.531947067,"t_done_unix":1779779633.5458903},{"request_id":"1313181:3:1364311:547","session_id":"1313181","input_length":59514,"t_proxy_recv":348750.565989665,"t_decision_unix":1779779615.5799308,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":24064,"affinity_cache_ratio":0.4043418355344961,"affinity_num_requests":1,"avg_num_requests":1.5,"fallback_score":35450,"tie_break_used":false,"cache_hit":24064,"estimated_new_tokens":35450,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":240128,"ongoing_decode_tokens":0,"pending_prefill_tokens":121856,"num_requests":3,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":24064,"new_prefill":35450,"score_linear":216064.0,"score_lmetric":471918},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":117888,"ongoing_decode_tokens":0,"pending_prefill_tokens":117888,"num_requests":1,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":59514,"score_linear":117888.0,"score_lmetric":177402},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":24763,"ongoing_decode_tokens":24763,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1539,"cache_hit":0,"new_prefill":59514,"score_linear":24763.0,"score_lmetric":119028},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76969,"ongoing_decode_tokens":0,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1458,"cache_hit":0,"new_prefill":59514,"score_linear":76969.0,"score_lmetric":136483},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90778,"ongoing_decode_tokens":0,"pending_prefill_tokens":63130,"num_requests":1,"active_p_offloads":0,"cached_blocks":1948,"cache_hit":0,"new_prefill":59514,"score_linear":90778.0,"score_lmetric":122644},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1131,"cache_hit":0,"new_prefill":59514,"score_linear":68259.0,"score_lmetric":59514},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":12636,"ongoing_decode_tokens":12636,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1150,"cache_hit":24064,"new_prefill":35450,"score_linear":-11428.0,"score_lmetric":35450},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":94143,"ongoing_decode_tokens":94143,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1011,"cache_hit":12800,"new_prefill":46714,"score_linear":81343.0,"score_lmetric":93428}],"chosen_score_linear":-11428.0,"chosen_score_lmetric":35450,"t_first_token":348760.63313588,"t_first_token_unix":1779779625.6470792,"t_done":348768.675828383,"t_done_unix":1779779633.6897712},{"request_id":"1363093:2:1368700:580","session_id":"1363093","input_length":63235,"t_proxy_recv":348766.1846645,"t_decision_unix":1779779631.1986058,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":62464,"affinity_cache_ratio":0.9878073851506286,"affinity_num_requests":2,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"cache_hit":62464,"estimated_new_tokens":771,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":240128,"ongoing_decode_tokens":0,"pending_prefill_tokens":121856,"num_requests":3,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":63235,"score_linear":240128.0,"score_lmetric":555273},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":132609,"ongoing_decode_tokens":0,"pending_prefill_tokens":119297,"num_requests":2,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":63235,"score_linear":132609.0,"score_lmetric":365064},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":8845,"ongoing_decode_tokens":8845,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1649,"cache_hit":0,"new_prefill":63235,"score_linear":8845.0,"score_lmetric":63235},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":72919,"ongoing_decode_tokens":0,"pending_prefill_tokens":72919,"num_requests":1,"active_p_offloads":0,"cached_blocks":1608,"cache_hit":0,"new_prefill":63235,"score_linear":72919.0,"score_lmetric":136154},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":112273,"ongoing_decode_tokens":90500,"pending_prefill_tokens":21773,"num_requests":2,"active_p_offloads":0,"cached_blocks":2070,"cache_hit":62464,"new_prefill":771,"score_linear":49809.0,"score_lmetric":45088},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":103453,"ongoing_decode_tokens":103453,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1149,"cache_hit":0,"new_prefill":63235,"score_linear":103453.0,"score_lmetric":126470},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":72498,"ongoing_decode_tokens":72498,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1219,"cache_hit":0,"new_prefill":63235,"score_linear":72498.0,"score_lmetric":126470},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1047,"cache_hit":0,"new_prefill":63235,"score_linear":80246.0,"score_lmetric":63235}],"chosen_score_linear":49809.0,"chosen_score_lmetric":45088,"t_first_token":348766.708307164,"t_first_token_unix":1779779631.7222502,"t_done":348768.999316191,"t_done_unix":1779779634.0132604},{"request_id":"1363943:2:1366428:556","session_id":"1363943","input_length":12984,"t_proxy_recv":348765.957813291,"t_decision_unix":1779779630.9717548,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":12800,"affinity_cache_ratio":0.9858287122612446,"affinity_num_requests":1,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"cache_hit":12800,"estimated_new_tokens":184,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":240128,"ongoing_decode_tokens":0,"pending_prefill_tokens":121856,"num_requests":3,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":8704,"new_prefill":4280,"score_linear":231424.0,"score_lmetric":378408},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":132609,"ongoing_decode_tokens":0,"pending_prefill_tokens":119297,"num_requests":2,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":12984,"score_linear":132609.0,"score_lmetric":264562},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":8845,"ongoing_decode_tokens":8845,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1649,"cache_hit":0,"new_prefill":12984,"score_linear":8845.0,"score_lmetric":12984},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":72919,"ongoing_decode_tokens":0,"pending_prefill_tokens":72919,"num_requests":1,"active_p_offloads":0,"cached_blocks":1608,"cache_hit":0,"new_prefill":12984,"score_linear":72919.0,"score_lmetric":85903},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":112273,"ongoing_decode_tokens":90500,"pending_prefill_tokens":21773,"num_requests":2,"active_p_offloads":0,"cached_blocks":2070,"cache_hit":4096,"new_prefill":8888,"score_linear":108177.0,"score_lmetric":61322},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":103453,"ongoing_decode_tokens":103453,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1149,"cache_hit":0,"new_prefill":12984,"score_linear":103453.0,"score_lmetric":25968},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":59514,"ongoing_decode_tokens":59514,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1219,"cache_hit":12800,"new_prefill":184,"score_linear":46714.0,"score_lmetric":184},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1047,"cache_hit":4096,"new_prefill":8888,"score_linear":76150.0,"score_lmetric":8888}],"chosen_score_linear":46714.0,"chosen_score_lmetric":184,"t_first_token":348766.028040913,"t_first_token_unix":1779779631.041984,"t_done":348769.158181007,"t_done_unix":1779779634.1721246},{"request_id":"1313181:4:1318843:342","session_id":"1313181","input_length":42704,"t_proxy_recv":348768.691293575,"t_decision_unix":1779779633.7052352,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":42496,"affinity_cache_ratio":0.9951292618958412,"affinity_num_requests":1,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":42496,"estimated_new_tokens":208,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":240128,"ongoing_decode_tokens":0,"pending_prefill_tokens":121856,"num_requests":3,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":24064,"new_prefill":18640,"score_linear":216064.0,"score_lmetric":421488},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":132609,"ongoing_decode_tokens":0,"pending_prefill_tokens":119297,"num_requests":2,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":42704,"score_linear":132609.0,"score_lmetric":324002},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1665,"cache_hit":0,"new_prefill":42704,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":72919,"ongoing_decode_tokens":0,"pending_prefill_tokens":72919,"num_requests":1,"active_p_offloads":0,"cached_blocks":1608,"cache_hit":0,"new_prefill":42704,"score_linear":72919.0,"score_lmetric":115623},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":112718,"ongoing_decode_tokens":112718,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2127,"cache_hit":0,"new_prefill":42704,"score_linear":112718.0,"score_lmetric":42704},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1154,"cache_hit":0,"new_prefill":42704,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":12984,"ongoing_decode_tokens":12984,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1223,"cache_hit":42496,"new_prefill":208,"score_linear":-29512.0,"score_lmetric":208},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1047,"cache_hit":12800,"new_prefill":29904,"score_linear":67446.0,"score_lmetric":29904}],"chosen_score_linear":-29512.0,"chosen_score_lmetric":208,"t_first_token":348768.764541446,"t_first_token_unix":1779779633.7784848,"t_done":348770.27482603,"t_done_unix":1779779635.2887697},{"request_id":"1340278:2:1367225:564","session_id":"1340278","input_length":49483,"t_proxy_recv":348761.058159557,"t_decision_unix":1779779626.0720987,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":48640,"affinity_cache_ratio":0.9829638461693915,"affinity_num_requests":2,"avg_num_requests":2.0,"fallback_score":null,"tie_break_used":false,"cache_hit":48640,"estimated_new_tokens":843,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":240128,"ongoing_decode_tokens":0,"pending_prefill_tokens":121856,"num_requests":3,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":49483,"score_linear":240128.0,"score_lmetric":514017},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":132609,"ongoing_decode_tokens":0,"pending_prefill_tokens":119297,"num_requests":2,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":49483,"score_linear":132609.0,"score_lmetric":337560},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":34226,"ongoing_decode_tokens":0,"pending_prefill_tokens":23474,"num_requests":2,"active_p_offloads":0,"cached_blocks":1592,"cache_hit":0,"new_prefill":49483,"score_linear":34226.0,"score_lmetric":145914},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76969,"ongoing_decode_tokens":0,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1458,"cache_hit":0,"new_prefill":49483,"score_linear":76969.0,"score_lmetric":126452},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":131795,"ongoing_decode_tokens":90778,"pending_prefill_tokens":7737,"num_requests":2,"active_p_offloads":0,"cached_blocks":1948,"cache_hit":48640,"new_prefill":843,"score_linear":83155.0,"score_lmetric":17160},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":139631,"ongoing_decode_tokens":103330,"pending_prefill_tokens":2509,"num_requests":3,"active_p_offloads":0,"cached_blocks":1138,"cache_hit":0,"new_prefill":49483,"score_linear":139631.0,"score_lmetric":155976},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":72150,"ongoing_decode_tokens":72150,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1150,"cache_hit":0,"new_prefill":49483,"score_linear":72150.0,"score_lmetric":98966},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1017,"cache_hit":0,"new_prefill":49483,"score_linear":80246.0,"score_lmetric":49483}],"chosen_score_linear":83155.0,"chosen_score_lmetric":17160,"t_first_token":348762.656345177,"t_first_token_unix":1779779627.670289,"t_done":348770.45277203,"t_done_unix":1779779635.4667156},{"request_id":"1369768:1:1369768:584","session_id":"1369768","input_length":7712,"t_proxy_recv":348769.872733836,"t_decision_unix":1779779634.8866754,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":7712,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":240128,"ongoing_decode_tokens":0,"pending_prefill_tokens":121856,"num_requests":3,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":7712,"score_linear":240128.0,"score_lmetric":388704},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":132609,"ongoing_decode_tokens":0,"pending_prefill_tokens":119297,"num_requests":2,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":7712,"score_linear":132609.0,"score_lmetric":254018},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1665,"cache_hit":0,"new_prefill":7712,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":72919,"ongoing_decode_tokens":0,"pending_prefill_tokens":72919,"num_requests":1,"active_p_offloads":0,"cached_blocks":1608,"cache_hit":0,"new_prefill":7712,"score_linear":72919.0,"score_lmetric":80631},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":49483,"ongoing_decode_tokens":49483,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":0,"new_prefill":7712,"score_linear":49483.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1154,"cache_hit":0,"new_prefill":7712,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":60864,"ongoing_decode_tokens":60864,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1223,"cache_hit":0,"new_prefill":7712,"score_linear":60864.0,"score_lmetric":15424},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1047,"cache_hit":0,"new_prefill":7712,"score_linear":80246.0,"score_lmetric":7712}],"chosen_score_linear":49483.0,"chosen_score_lmetric":0,"t_first_token":348770.417652323,"t_first_token_unix":1779779635.4315958,"t_done":348770.68837873,"t_done_unix":1779779635.7023218},{"request_id":"1340278:3:1368072:576","session_id":"1340278","input_length":50124,"t_proxy_recv":348770.48503542,"t_decision_unix":1779779635.4989765,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":49664,"affinity_cache_ratio":0.9908227595563004,"affinity_num_requests":0,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":49664,"estimated_new_tokens":460,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":240128,"ongoing_decode_tokens":0,"pending_prefill_tokens":121856,"num_requests":3,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":50124,"score_linear":240128.0,"score_lmetric":515940},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":132609,"ongoing_decode_tokens":0,"pending_prefill_tokens":119297,"num_requests":2,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":50124,"score_linear":132609.0,"score_lmetric":338842},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":23307,"ongoing_decode_tokens":0,"pending_prefill_tokens":7947,"num_requests":1,"active_p_offloads":0,"cached_blocks":1665,"cache_hit":0,"new_prefill":50124,"score_linear":23307.0,"score_lmetric":58071},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":72919,"ongoing_decode_tokens":0,"pending_prefill_tokens":72919,"num_requests":1,"active_p_offloads":0,"cached_blocks":1608,"cache_hit":0,"new_prefill":50124,"score_linear":72919.0,"score_lmetric":123043},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":7712,"ongoing_decode_tokens":7712,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2130,"cache_hit":49664,"new_prefill":460,"score_linear":-41952.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1154,"cache_hit":0,"new_prefill":50124,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":78164,"ongoing_decode_tokens":18160,"pending_prefill_tokens":17508,"num_requests":2,"active_p_offloads":0,"cached_blocks":1223,"cache_hit":0,"new_prefill":50124,"score_linear":78164.0,"score_lmetric":135264},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1047,"cache_hit":0,"new_prefill":50124,"score_linear":80246.0,"score_lmetric":50124}],"chosen_score_linear":-41952.0,"chosen_score_lmetric":0,"t_first_token":348770.662744415,"t_first_token_unix":1779779635.6766877,"t_done":348770.689442017,"t_done_unix":1779779635.7033849},{"request_id":"1366128:3:1369902:585","session_id":"1366128","input_length":23307,"t_proxy_recv":348770.340430071,"t_decision_unix":1779779635.3543713,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":15360,"affinity_cache_ratio":0.6590294761230532,"affinity_num_requests":0,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":15360,"estimated_new_tokens":7947,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":240128,"ongoing_decode_tokens":0,"pending_prefill_tokens":121856,"num_requests":3,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":23307,"score_linear":240128.0,"score_lmetric":435489},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":132609,"ongoing_decode_tokens":0,"pending_prefill_tokens":119297,"num_requests":2,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":23307,"score_linear":132609.0,"score_lmetric":285208},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1665,"cache_hit":15360,"new_prefill":7947,"score_linear":-15360.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":72919,"ongoing_decode_tokens":0,"pending_prefill_tokens":72919,"num_requests":1,"active_p_offloads":0,"cached_blocks":1608,"cache_hit":0,"new_prefill":23307,"score_linear":72919.0,"score_lmetric":96226},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":57195,"ongoing_decode_tokens":49483,"pending_prefill_tokens":7712,"num_requests":1,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":0,"new_prefill":23307,"score_linear":57195.0,"score_lmetric":31019},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1154,"cache_hit":0,"new_prefill":23307,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":78164,"ongoing_decode_tokens":18160,"pending_prefill_tokens":17508,"num_requests":2,"active_p_offloads":0,"cached_blocks":1223,"cache_hit":0,"new_prefill":23307,"score_linear":78164.0,"score_lmetric":81630},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1047,"cache_hit":0,"new_prefill":23307,"score_linear":80246.0,"score_lmetric":23307}],"chosen_score_linear":-15360.0,"chosen_score_lmetric":0,"t_first_token":348771.58014292,"t_first_token_unix":1779779636.5940862,"t_done":348772.002199923,"t_done_unix":1779779637.016143},{"request_id":"1344773:13:1370257:589","session_id":"1344773","input_length":35413,"t_proxy_recv":348771.543908263,"t_decision_unix":1779779636.55785,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":35328,"affinity_cache_ratio":0.9975997515036851,"affinity_num_requests":1,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":35328,"estimated_new_tokens":85,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":240128,"ongoing_decode_tokens":0,"pending_prefill_tokens":121856,"num_requests":3,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":35413,"score_linear":240128.0,"score_lmetric":471807},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":132609,"ongoing_decode_tokens":0,"pending_prefill_tokens":119297,"num_requests":2,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":35413,"score_linear":132609.0,"score_lmetric":309420},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":23307,"ongoing_decode_tokens":0,"pending_prefill_tokens":7947,"num_requests":1,"active_p_offloads":0,"cached_blocks":1665,"cache_hit":0,"new_prefill":35413,"score_linear":23307.0,"score_lmetric":43360},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":72919,"ongoing_decode_tokens":0,"pending_prefill_tokens":72919,"num_requests":1,"active_p_offloads":0,"cached_blocks":1608,"cache_hit":0,"new_prefill":35413,"score_linear":72919.0,"score_lmetric":108332},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":47802,"ongoing_decode_tokens":0,"pending_prefill_tokens":18618,"num_requests":1,"active_p_offloads":0,"cached_blocks":2145,"cache_hit":0,"new_prefill":35413,"score_linear":47802.0,"score_lmetric":54031},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":11133,"ongoing_decode_tokens":0,"pending_prefill_tokens":8061,"num_requests":1,"active_p_offloads":0,"cached_blocks":1154,"cache_hit":35328,"new_prefill":85,"score_linear":-24195.0,"score_lmetric":8146},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":78164,"ongoing_decode_tokens":18160,"pending_prefill_tokens":17508,"num_requests":2,"active_p_offloads":0,"cached_blocks":1223,"cache_hit":0,"new_prefill":35413,"score_linear":78164.0,"score_lmetric":105842},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1047,"cache_hit":0,"new_prefill":35413,"score_linear":80246.0,"score_lmetric":35413}],"chosen_score_linear":-24195.0,"chosen_score_lmetric":8146,"t_first_token":348772.079593081,"t_first_token_unix":1779779637.0935366,"t_done":348772.109102319,"t_done_unix":1779779637.1230454},{"request_id":"1367424:2:1370211:588","session_id":"1367424","input_length":11133,"t_proxy_recv":348771.348383656,"t_decision_unix":1779779636.3623252,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":3072,"affinity_cache_ratio":0.27593640528159524,"affinity_num_requests":0,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":false,"cache_hit":3072,"estimated_new_tokens":8061,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":240128,"ongoing_decode_tokens":0,"pending_prefill_tokens":121856,"num_requests":3,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":11133,"score_linear":240128.0,"score_lmetric":398967},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":132609,"ongoing_decode_tokens":0,"pending_prefill_tokens":119297,"num_requests":2,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":11133,"score_linear":132609.0,"score_lmetric":260860},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":23307,"ongoing_decode_tokens":0,"pending_prefill_tokens":7947,"num_requests":1,"active_p_offloads":0,"cached_blocks":1665,"cache_hit":0,"new_prefill":11133,"score_linear":23307.0,"score_lmetric":19080},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":72919,"ongoing_decode_tokens":0,"pending_prefill_tokens":72919,"num_requests":1,"active_p_offloads":0,"cached_blocks":1608,"cache_hit":0,"new_prefill":11133,"score_linear":72919.0,"score_lmetric":84052},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":47802,"ongoing_decode_tokens":0,"pending_prefill_tokens":18618,"num_requests":1,"active_p_offloads":0,"cached_blocks":2145,"cache_hit":0,"new_prefill":11133,"score_linear":47802.0,"score_lmetric":29751},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1154,"cache_hit":3072,"new_prefill":8061,"score_linear":-3072.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":78164,"ongoing_decode_tokens":18160,"pending_prefill_tokens":17508,"num_requests":2,"active_p_offloads":0,"cached_blocks":1223,"cache_hit":0,"new_prefill":11133,"score_linear":78164.0,"score_lmetric":57282},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1047,"cache_hit":0,"new_prefill":11133,"score_linear":80246.0,"score_lmetric":11133}],"chosen_score_linear":-3072.0,"chosen_score_lmetric":0,"t_first_token":348772.032615139,"t_first_token_unix":1779779637.0465584,"t_done":348772.135587402,"t_done_unix":1779779637.1495304},{"request_id":"1370269:1:1370269:591","session_id":"1370269","input_length":2381,"t_proxy_recv":348771.648318096,"t_decision_unix":1779779636.66226,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.75,"fallback_score":2381,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":2381,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":304659,"ongoing_decode_tokens":0,"pending_prefill_tokens":127507,"num_requests":4,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":2381,"score_linear":304659.0,"score_lmetric":519552},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":132609,"ongoing_decode_tokens":0,"pending_prefill_tokens":119297,"num_requests":2,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":2381,"score_linear":132609.0,"score_lmetric":243356},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":23307,"ongoing_decode_tokens":23307,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1665,"cache_hit":0,"new_prefill":2381,"score_linear":23307.0,"score_lmetric":2381},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":72919,"ongoing_decode_tokens":0,"pending_prefill_tokens":72919,"num_requests":1,"active_p_offloads":0,"cached_blocks":1608,"cache_hit":0,"new_prefill":2381,"score_linear":72919.0,"score_lmetric":75300},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":47802,"ongoing_decode_tokens":0,"pending_prefill_tokens":18618,"num_requests":1,"active_p_offloads":0,"cached_blocks":2145,"cache_hit":0,"new_prefill":2381,"score_linear":47802.0,"score_lmetric":20999},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":46546,"ongoing_decode_tokens":0,"pending_prefill_tokens":8146,"num_requests":2,"active_p_offloads":0,"cached_blocks":1154,"cache_hit":0,"new_prefill":2381,"score_linear":46546.0,"score_lmetric":21054},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":78164,"ongoing_decode_tokens":18160,"pending_prefill_tokens":17508,"num_requests":2,"active_p_offloads":0,"cached_blocks":1223,"cache_hit":0,"new_prefill":2381,"score_linear":78164.0,"score_lmetric":39778},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1047,"cache_hit":0,"new_prefill":2381,"score_linear":80246.0,"score_lmetric":2381}],"chosen_score_linear":23307.0,"chosen_score_lmetric":2381,"t_first_token":348771.82700971,"t_first_token_unix":1779779636.8409536,"t_done":348772.329589422,"t_done_unix":1779779637.3435326},{"request_id":"1370466:1:1370466:592","session_id":"1370466","input_length":7710,"t_proxy_recv":348772.400016941,"t_decision_unix":1779779637.413958,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":7710,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":304659,"ongoing_decode_tokens":0,"pending_prefill_tokens":127507,"num_requests":4,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":7710,"score_linear":304659.0,"score_lmetric":540868},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":132609,"ongoing_decode_tokens":0,"pending_prefill_tokens":119297,"num_requests":2,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":7710,"score_linear":132609.0,"score_lmetric":254014},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1684,"cache_hit":0,"new_prefill":7710,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":72919,"ongoing_decode_tokens":0,"pending_prefill_tokens":72919,"num_requests":1,"active_p_offloads":0,"cached_blocks":1608,"cache_hit":0,"new_prefill":7710,"score_linear":72919.0,"score_lmetric":80629},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":47802,"ongoing_decode_tokens":0,"pending_prefill_tokens":18618,"num_requests":1,"active_p_offloads":0,"cached_blocks":2145,"cache_hit":0,"new_prefill":7710,"score_linear":47802.0,"score_lmetric":26328},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1169,"cache_hit":0,"new_prefill":7710,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":78164,"ongoing_decode_tokens":18160,"pending_prefill_tokens":17508,"num_requests":2,"active_p_offloads":0,"cached_blocks":1223,"cache_hit":0,"new_prefill":7710,"score_linear":78164.0,"score_lmetric":50436},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1047,"cache_hit":0,"new_prefill":7710,"score_linear":80246.0,"score_lmetric":7710}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348772.92995892,"t_first_token_unix":1779779637.943903,"t_done":348773.045012978,"t_done_unix":1779779638.0589566},{"request_id":"1368070:2:1370108:586","session_id":"1368070","input_length":29501,"t_proxy_recv":348771.054897841,"t_decision_unix":1779779636.068839,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":21504,"affinity_cache_ratio":0.7289244432392122,"affinity_num_requests":-1,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":21504,"estimated_new_tokens":7997,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":240128,"ongoing_decode_tokens":0,"pending_prefill_tokens":121856,"num_requests":3,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":29501,"score_linear":240128.0,"score_lmetric":454071},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":132609,"ongoing_decode_tokens":0,"pending_prefill_tokens":119297,"num_requests":2,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":29501,"score_linear":132609.0,"score_lmetric":297596},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":23307,"ongoing_decode_tokens":0,"pending_prefill_tokens":7947,"num_requests":1,"active_p_offloads":0,"cached_blocks":1665,"cache_hit":0,"new_prefill":29501,"score_linear":23307.0,"score_lmetric":37448},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":72919,"ongoing_decode_tokens":0,"pending_prefill_tokens":72919,"num_requests":1,"active_p_offloads":0,"cached_blocks":1608,"cache_hit":0,"new_prefill":29501,"score_linear":72919.0,"score_lmetric":102420},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":2145,"cache_hit":21504,"new_prefill":7997,"score_linear":-21504.0,"score_lmetric":-7997},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1154,"cache_hit":0,"new_prefill":29501,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":78164,"ongoing_decode_tokens":18160,"pending_prefill_tokens":17508,"num_requests":2,"active_p_offloads":0,"cached_blocks":1223,"cache_hit":0,"new_prefill":29501,"score_linear":78164.0,"score_lmetric":94018},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1047,"cache_hit":0,"new_prefill":29501,"score_linear":80246.0,"score_lmetric":29501}],"chosen_score_linear":-21504.0,"chosen_score_lmetric":-7997,"t_first_token":348772.557591617,"t_first_token_unix":1779779637.571535,"t_done":348774.525016613,"t_done_unix":1779779639.53896},{"request_id":"1277428:6:1371101:593","session_id":"1277428","input_length":36583,"t_proxy_recv":348774.429958353,"t_decision_unix":1779779639.4438994,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":36352,"affinity_cache_ratio":0.9936855916682612,"affinity_num_requests":0,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":36352,"estimated_new_tokens":231,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":304659,"ongoing_decode_tokens":240128,"pending_prefill_tokens":5651,"num_requests":4,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":36583,"score_linear":304659.0,"score_lmetric":168936},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":132609,"ongoing_decode_tokens":0,"pending_prefill_tokens":119297,"num_requests":2,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":4608,"new_prefill":31975,"score_linear":128001.0,"score_lmetric":302544},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1684,"cache_hit":0,"new_prefill":36583,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":72919,"ongoing_decode_tokens":0,"pending_prefill_tokens":72919,"num_requests":1,"active_p_offloads":0,"cached_blocks":1608,"cache_hit":17408,"new_prefill":19175,"score_linear":55511.0,"score_lmetric":92094},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":47802,"ongoing_decode_tokens":47802,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2145,"cache_hit":0,"new_prefill":36583,"score_linear":47802.0,"score_lmetric":36583},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1184,"cache_hit":36352,"new_prefill":231,"score_linear":-36352.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":78164,"ongoing_decode_tokens":18160,"pending_prefill_tokens":17508,"num_requests":2,"active_p_offloads":0,"cached_blocks":1223,"cache_hit":0,"new_prefill":36583,"score_linear":78164.0,"score_lmetric":108182},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1047,"cache_hit":0,"new_prefill":36583,"score_linear":80246.0,"score_lmetric":36583}],"chosen_score_linear":-36352.0,"chosen_score_lmetric":0,"t_first_token":348774.564085312,"t_first_token_unix":1779779639.5780284,"t_done":348774.894343385,"t_done_unix":1779779639.9082866},{"request_id":"1371337:1:1371337:597","session_id":"1371337","input_length":9399,"t_proxy_recv":348775.265374531,"t_decision_unix":1779779640.2793157,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":9399,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":304659,"ongoing_decode_tokens":240128,"pending_prefill_tokens":5651,"num_requests":4,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":9399,"score_linear":304659.0,"score_lmetric":60200},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":132609,"ongoing_decode_tokens":0,"pending_prefill_tokens":119297,"num_requests":2,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":9399,"score_linear":132609.0,"score_lmetric":257392},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1684,"cache_hit":0,"new_prefill":9399,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":72919,"ongoing_decode_tokens":0,"pending_prefill_tokens":72919,"num_requests":1,"active_p_offloads":0,"cached_blocks":1608,"cache_hit":0,"new_prefill":9399,"score_linear":72919.0,"score_lmetric":82318},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":18301,"ongoing_decode_tokens":18301,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2160,"cache_hit":0,"new_prefill":9399,"score_linear":18301.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1184,"cache_hit":0,"new_prefill":9399,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":110978,"ongoing_decode_tokens":18160,"pending_prefill_tokens":23698,"num_requests":3,"active_p_offloads":0,"cached_blocks":1223,"cache_hit":0,"new_prefill":9399,"score_linear":110978.0,"score_lmetric":99291},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1047,"cache_hit":0,"new_prefill":9399,"score_linear":80246.0,"score_lmetric":9399}],"chosen_score_linear":18301.0,"chosen_score_lmetric":0,"t_first_token":348775.97621313,"t_first_token_unix":1779779640.990157,"t_done":348776.661082236,"t_done_unix":1779779641.6750257},{"request_id":"1371622:1:1371622:598","session_id":"1371622","input_length":13616,"t_proxy_recv":348776.268575199,"t_decision_unix":1779779641.2825162,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.5,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":13616,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":304659,"ongoing_decode_tokens":240128,"pending_prefill_tokens":5651,"num_requests":4,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":13616,"score_linear":304659.0,"score_lmetric":77068},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":132609,"ongoing_decode_tokens":0,"pending_prefill_tokens":119297,"num_requests":2,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":13616,"score_linear":132609.0,"score_lmetric":265826},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1684,"cache_hit":0,"new_prefill":13616,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":72919,"ongoing_decode_tokens":0,"pending_prefill_tokens":72919,"num_requests":1,"active_p_offloads":0,"cached_blocks":1608,"cache_hit":0,"new_prefill":13616,"score_linear":72919.0,"score_lmetric":86535},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27700,"ongoing_decode_tokens":27700,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2160,"cache_hit":11264,"new_prefill":2352,"score_linear":16436.0,"score_lmetric":2352},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1184,"cache_hit":0,"new_prefill":13616,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":110978,"ongoing_decode_tokens":78164,"pending_prefill_tokens":6190,"num_requests":3,"active_p_offloads":0,"cached_blocks":1223,"cache_hit":0,"new_prefill":13616,"score_linear":110978.0,"score_lmetric":59418},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1047,"cache_hit":0,"new_prefill":13616,"score_linear":80246.0,"score_lmetric":13616}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348777.431962792,"t_first_token_unix":1779779642.4459062,"t_done":348778.17607294,"t_done_unix":1779779643.190016},{"request_id":"1367329:2:1371226:595","session_id":"1367329","input_length":32814,"t_proxy_recv":348774.837534353,"t_decision_unix":1779779639.8514752,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":26624,"affinity_cache_ratio":0.8113610044493205,"affinity_num_requests":2,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":26624,"estimated_new_tokens":6190,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":304659,"ongoing_decode_tokens":240128,"pending_prefill_tokens":5651,"num_requests":4,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":32814,"score_linear":304659.0,"score_lmetric":153860},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":132609,"ongoing_decode_tokens":0,"pending_prefill_tokens":119297,"num_requests":2,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":32814,"score_linear":132609.0,"score_lmetric":304222},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1684,"cache_hit":0,"new_prefill":32814,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":72919,"ongoing_decode_tokens":0,"pending_prefill_tokens":72919,"num_requests":1,"active_p_offloads":0,"cached_blocks":1608,"cache_hit":0,"new_prefill":32814,"score_linear":72919.0,"score_lmetric":105733},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":18301,"ongoing_decode_tokens":18301,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2160,"cache_hit":0,"new_prefill":32814,"score_linear":18301.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":36583,"ongoing_decode_tokens":36583,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1184,"cache_hit":0,"new_prefill":32814,"score_linear":36583.0,"score_lmetric":32814},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":78164,"ongoing_decode_tokens":18160,"pending_prefill_tokens":17508,"num_requests":2,"active_p_offloads":0,"cached_blocks":1223,"cache_hit":26624,"new_prefill":6190,"score_linear":51540.0,"score_lmetric":47396},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1047,"cache_hit":0,"new_prefill":32814,"score_linear":80246.0,"score_lmetric":32814}],"chosen_score_linear":51540.0,"chosen_score_lmetric":47396,"t_first_token":348777.475852037,"t_first_token_unix":1779779642.4897957,"t_done":348778.899183149,"t_done_unix":1779779643.9131265},{"request_id":"1366128:4:1371629:599","session_id":"1366128","input_length":35037,"t_proxy_recv":348776.292472941,"t_decision_unix":1779779641.3064146,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":23040,"affinity_cache_ratio":0.6575905471358849,"affinity_num_requests":0,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"cache_hit":23040,"estimated_new_tokens":11997,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":304659,"ongoing_decode_tokens":240128,"pending_prefill_tokens":5651,"num_requests":4,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":35037,"score_linear":304659.0,"score_lmetric":162752},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":132609,"ongoing_decode_tokens":0,"pending_prefill_tokens":119297,"num_requests":2,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":35037,"score_linear":132609.0,"score_lmetric":308668},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1684,"cache_hit":23040,"new_prefill":11997,"score_linear":-23040.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":72919,"ongoing_decode_tokens":0,"pending_prefill_tokens":72919,"num_requests":1,"active_p_offloads":0,"cached_blocks":1608,"cache_hit":0,"new_prefill":35037,"score_linear":72919.0,"score_lmetric":107956},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":27700,"ongoing_decode_tokens":27700,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2160,"cache_hit":0,"new_prefill":35037,"score_linear":27700.0,"score_lmetric":35037},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":13616,"ongoing_decode_tokens":0,"pending_prefill_tokens":13616,"num_requests":1,"active_p_offloads":0,"cached_blocks":1184,"cache_hit":0,"new_prefill":35037,"score_linear":13616.0,"score_lmetric":48653},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":110978,"ongoing_decode_tokens":78164,"pending_prefill_tokens":6190,"num_requests":3,"active_p_offloads":0,"cached_blocks":1223,"cache_hit":0,"new_prefill":35037,"score_linear":110978.0,"score_lmetric":123681},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1047,"cache_hit":0,"new_prefill":35037,"score_linear":80246.0,"score_lmetric":35037}],"chosen_score_linear":-23040.0,"chosen_score_lmetric":0,"t_first_token":348778.818125733,"t_first_token_unix":1779779643.8320692,"t_done":348779.080516982,"t_done_unix":1779779644.0944602},{"request_id":"1363943:3:1369535:582","session_id":"1363943","input_length":18160,"t_proxy_recv":348769.164742865,"t_decision_unix":1779779634.1786847,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":12800,"affinity_cache_ratio":0.7048458149779736,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":12800,"estimated_new_tokens":5360,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":240128,"ongoing_decode_tokens":0,"pending_prefill_tokens":121856,"num_requests":3,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":8704,"new_prefill":9456,"score_linear":231424.0,"score_lmetric":393936},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":132609,"ongoing_decode_tokens":0,"pending_prefill_tokens":119297,"num_requests":2,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":18160,"score_linear":132609.0,"score_lmetric":274914},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1665,"cache_hit":0,"new_prefill":18160,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":72919,"ongoing_decode_tokens":0,"pending_prefill_tokens":72919,"num_requests":1,"active_p_offloads":0,"cached_blocks":1608,"cache_hit":0,"new_prefill":18160,"score_linear":72919.0,"score_lmetric":91079},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":49483,"ongoing_decode_tokens":49483,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":4096,"new_prefill":14064,"score_linear":45387.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1154,"cache_hit":0,"new_prefill":18160,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":42704,"ongoing_decode_tokens":42704,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1223,"cache_hit":12800,"new_prefill":5360,"score_linear":29904.0,"score_lmetric":5360},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1047,"cache_hit":4096,"new_prefill":14064,"score_linear":76150.0,"score_lmetric":14064}],"chosen_score_linear":29904.0,"chosen_score_lmetric":5360,"t_first_token":348769.858862426,"t_first_token_unix":1779779634.872806,"t_done":348779.568481235,"t_done_unix":1779779644.5824246},{"request_id":"1371929:1:1371929:601","session_id":"1371929","input_length":17407,"t_proxy_recv":348777.374743232,"t_decision_unix":1779779642.3886845,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.75,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":17407,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":304659,"ongoing_decode_tokens":240128,"pending_prefill_tokens":5651,"num_requests":4,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":17407,"score_linear":304659.0,"score_lmetric":92232},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":132609,"ongoing_decode_tokens":0,"pending_prefill_tokens":119297,"num_requests":2,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":17407,"score_linear":132609.0,"score_lmetric":273408},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":35037,"ongoing_decode_tokens":0,"pending_prefill_tokens":11997,"num_requests":1,"active_p_offloads":0,"cached_blocks":1684,"cache_hit":0,"new_prefill":17407,"score_linear":35037.0,"score_lmetric":29404},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":72919,"ongoing_decode_tokens":0,"pending_prefill_tokens":72919,"num_requests":1,"active_p_offloads":0,"cached_blocks":1608,"cache_hit":0,"new_prefill":17407,"score_linear":72919.0,"score_lmetric":90326},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":18301,"ongoing_decode_tokens":18301,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2178,"cache_hit":0,"new_prefill":17407,"score_linear":18301.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":13616,"ongoing_decode_tokens":0,"pending_prefill_tokens":13616,"num_requests":1,"active_p_offloads":0,"cached_blocks":1184,"cache_hit":0,"new_prefill":17407,"score_linear":13616.0,"score_lmetric":31023},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":110978,"ongoing_decode_tokens":78164,"pending_prefill_tokens":6190,"num_requests":3,"active_p_offloads":0,"cached_blocks":1223,"cache_hit":0,"new_prefill":17407,"score_linear":110978.0,"score_lmetric":70791},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":106951,"ongoing_decode_tokens":80246,"pending_prefill_tokens":12369,"num_requests":2,"active_p_offloads":0,"cached_blocks":1047,"cache_hit":0,"new_prefill":17407,"score_linear":106951.0,"score_lmetric":59552}],"chosen_score_linear":18301.0,"chosen_score_lmetric":0,"t_first_token":348779.079756231,"t_first_token_unix":1779779644.0936992,"t_done":348779.712880149,"t_done_unix":1779779644.7268229},{"request_id":"1370129:1:1370129:587","session_id":"1370129","input_length":18301,"t_proxy_recv":348771.126391821,"t_decision_unix":1779779636.1403334,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":false,"cache_hit":7680,"estimated_new_tokens":10621,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":240128,"ongoing_decode_tokens":0,"pending_prefill_tokens":121856,"num_requests":3,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":7680,"new_prefill":10621,"score_linear":232448.0,"score_lmetric":397431},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":132609,"ongoing_decode_tokens":0,"pending_prefill_tokens":119297,"num_requests":2,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":18301,"score_linear":132609.0,"score_lmetric":275196},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":23307,"ongoing_decode_tokens":0,"pending_prefill_tokens":7947,"num_requests":1,"active_p_offloads":0,"cached_blocks":1665,"cache_hit":0,"new_prefill":18301,"score_linear":23307.0,"score_lmetric":26248},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":72919,"ongoing_decode_tokens":0,"pending_prefill_tokens":72919,"num_requests":1,"active_p_offloads":0,"cached_blocks":1608,"cache_hit":0,"new_prefill":18301,"score_linear":72919.0,"score_lmetric":91220},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":29501,"ongoing_decode_tokens":0,"pending_prefill_tokens":7997,"num_requests":0,"active_p_offloads":0,"cached_blocks":2145,"cache_hit":7680,"new_prefill":10621,"score_linear":21821.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1154,"cache_hit":0,"new_prefill":18301,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":78164,"ongoing_decode_tokens":18160,"pending_prefill_tokens":17508,"num_requests":2,"active_p_offloads":0,"cached_blocks":1223,"cache_hit":7680,"new_prefill":10621,"score_linear":70484.0,"score_lmetric":56258},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1047,"cache_hit":7680,"new_prefill":10621,"score_linear":72566.0,"score_lmetric":10621}],"chosen_score_linear":21821.0,"chosen_score_lmetric":0,"t_first_token":348774.359100451,"t_first_token_unix":1779779639.3730438,"t_done":348780.090880946,"t_done_unix":1779779645.1048243},{"request_id":"1363943:4:1372250:603","session_id":"1363943","input_length":18396,"t_proxy_recv":348779.573222997,"t_decision_unix":1779779644.5871646,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":17920,"affinity_cache_ratio":0.974124809741248,"affinity_num_requests":1,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":17920,"estimated_new_tokens":476,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":304659,"ongoing_decode_tokens":240128,"pending_prefill_tokens":5651,"num_requests":4,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":8704,"new_prefill":9692,"score_linear":295955.0,"score_lmetric":61372},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":132609,"ongoing_decode_tokens":0,"pending_prefill_tokens":119297,"num_requests":2,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":18396,"score_linear":132609.0,"score_lmetric":275386},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1707,"cache_hit":0,"new_prefill":18396,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":72919,"ongoing_decode_tokens":0,"pending_prefill_tokens":72919,"num_requests":1,"active_p_offloads":0,"cached_blocks":1608,"cache_hit":0,"new_prefill":18396,"score_linear":72919.0,"score_lmetric":91315},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":35708,"ongoing_decode_tokens":35708,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2178,"cache_hit":4096,"new_prefill":14300,"score_linear":31612.0,"score_lmetric":14300},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":38252,"ongoing_decode_tokens":38252,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1210,"cache_hit":0,"new_prefill":18396,"score_linear":38252.0,"score_lmetric":18396},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":60004,"ongoing_decode_tokens":60004,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1245,"cache_hit":17920,"new_prefill":476,"score_linear":42084.0,"score_lmetric":476},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":106951,"ongoing_decode_tokens":106951,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1047,"cache_hit":4096,"new_prefill":14300,"score_linear":102855.0,"score_lmetric":28600}],"chosen_score_linear":42084.0,"chosen_score_lmetric":476,"t_first_token":348779.619929719,"t_first_token_unix":1779779644.633873,"t_done":348780.492266516,"t_done_unix":1779779645.5062122},{"request_id":"1372791:1:1372791:604","session_id":"1372791","input_length":4811,"t_proxy_recv":348780.380258005,"t_decision_unix":1779779645.3941994,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.375,"fallback_score":-4811,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":4811,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":304659,"ongoing_decode_tokens":240128,"pending_prefill_tokens":5651,"num_requests":4,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":4811,"score_linear":304659.0,"score_lmetric":41848},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":132609,"ongoing_decode_tokens":0,"pending_prefill_tokens":119297,"num_requests":2,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":4811,"score_linear":132609.0,"score_lmetric":248216},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1707,"cache_hit":0,"new_prefill":4811,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":72919,"ongoing_decode_tokens":0,"pending_prefill_tokens":72919,"num_requests":1,"active_p_offloads":0,"cached_blocks":1608,"cache_hit":0,"new_prefill":4811,"score_linear":72919.0,"score_lmetric":77730},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":2233,"cache_hit":0,"new_prefill":4811,"score_linear":0.0,"score_lmetric":-4811},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":38252,"ongoing_decode_tokens":38252,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1210,"cache_hit":0,"new_prefill":4811,"score_linear":38252.0,"score_lmetric":4811},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":78400,"ongoing_decode_tokens":78400,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1245,"cache_hit":0,"new_prefill":4811,"score_linear":78400.0,"score_lmetric":9622},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":106951,"ongoing_decode_tokens":106951,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1047,"cache_hit":0,"new_prefill":4811,"score_linear":106951.0,"score_lmetric":9622}],"chosen_score_linear":0.0,"chosen_score_lmetric":-4811,"t_first_token":348780.684132579,"t_first_token_unix":1779779645.6980762,"t_done":348780.865264148,"t_done_unix":1779779645.8792076},{"request_id":"1367957:1:1367957:572","session_id":"1367957","input_length":72919,"t_proxy_recv":348763.591103444,"t_decision_unix":1779779628.605045,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.375,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":72919,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":240128,"ongoing_decode_tokens":0,"pending_prefill_tokens":121856,"num_requests":3,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":72919,"score_linear":240128.0,"score_lmetric":584325},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":132609,"ongoing_decode_tokens":0,"pending_prefill_tokens":119297,"num_requests":2,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":72919,"score_linear":132609.0,"score_lmetric":384432},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":53455,"ongoing_decode_tokens":53455,"pending_prefill_tokens":0,"num_requests":4,"active_p_offloads":0,"cached_blocks":1592,"cache_hit":18944,"new_prefill":53975,"score_linear":34511.0,"score_lmetric":215900},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1608,"cache_hit":0,"new_prefill":72919,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":153185,"ongoing_decode_tokens":153185,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1948,"cache_hit":0,"new_prefill":72919,"score_linear":153185.0,"score_lmetric":145838},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":143023,"ongoing_decode_tokens":139631,"pending_prefill_tokens":3392,"num_requests":4,"active_p_offloads":0,"cached_blocks":1138,"cache_hit":0,"new_prefill":72919,"score_linear":143023.0,"score_lmetric":305244},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":98987,"ongoing_decode_tokens":72150,"pending_prefill_tokens":26837,"num_requests":3,"active_p_offloads":0,"cached_blocks":1150,"cache_hit":0,"new_prefill":72919,"score_linear":98987.0,"score_lmetric":299268},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1043,"cache_hit":0,"new_prefill":72919,"score_linear":80246.0,"score_lmetric":72919}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348781.824547889,"t_first_token_unix":1779779646.8384917,"t_done":348782.50678111,"t_done_unix":1779779647.5207248},{"request_id":"1362265:3:1373357:605","session_id":"1362265","input_length":10284,"t_proxy_recv":348782.383077706,"t_decision_unix":1779779647.397019,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":9728,"affinity_cache_ratio":0.9459354336833917,"affinity_num_requests":0,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":9728,"estimated_new_tokens":556,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":304659,"ongoing_decode_tokens":240128,"pending_prefill_tokens":5651,"num_requests":4,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":10284,"score_linear":304659.0,"score_lmetric":63740},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":132609,"ongoing_decode_tokens":0,"pending_prefill_tokens":119297,"num_requests":2,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":10284,"score_linear":132609.0,"score_lmetric":259162},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1707,"cache_hit":9728,"new_prefill":556,"score_linear":-9728.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":72919,"ongoing_decode_tokens":72919,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1608,"cache_hit":0,"new_prefill":10284,"score_linear":72919.0,"score_lmetric":10284},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":10284,"score_linear":0.0,"score_lmetric":-10284},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":38252,"ongoing_decode_tokens":38252,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1210,"cache_hit":0,"new_prefill":10284,"score_linear":38252.0,"score_lmetric":10284},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":60004,"ongoing_decode_tokens":60004,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1246,"cache_hit":0,"new_prefill":10284,"score_linear":60004.0,"score_lmetric":10284},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":106951,"ongoing_decode_tokens":106951,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1047,"cache_hit":0,"new_prefill":10284,"score_linear":106951.0,"score_lmetric":20568}],"chosen_score_linear":-9728.0,"chosen_score_lmetric":0,"t_first_token":348782.434105007,"t_first_token_unix":1779779647.4480484,"t_done":348782.53742707,"t_done_unix":1779779647.5513704},{"request_id":"1277428:7:1372183:602","session_id":"1277428","input_length":38252,"t_proxy_recv":348778.342968408,"t_decision_unix":1779779643.3569098,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":36352,"affinity_cache_ratio":0.9503293945414619,"affinity_num_requests":0,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"cache_hit":36352,"estimated_new_tokens":1900,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":304659,"ongoing_decode_tokens":240128,"pending_prefill_tokens":5651,"num_requests":4,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":38252,"score_linear":304659.0,"score_lmetric":175612},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":132609,"ongoing_decode_tokens":0,"pending_prefill_tokens":119297,"num_requests":2,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":4608,"new_prefill":33644,"score_linear":128001.0,"score_lmetric":305882},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":35037,"ongoing_decode_tokens":0,"pending_prefill_tokens":11997,"num_requests":1,"active_p_offloads":0,"cached_blocks":1684,"cache_hit":0,"new_prefill":38252,"score_linear":35037.0,"score_lmetric":50249},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":72919,"ongoing_decode_tokens":0,"pending_prefill_tokens":72919,"num_requests":1,"active_p_offloads":0,"cached_blocks":1608,"cache_hit":17408,"new_prefill":20844,"score_linear":55511.0,"score_lmetric":93763},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":35708,"ongoing_decode_tokens":18301,"pending_prefill_tokens":17407,"num_requests":1,"active_p_offloads":0,"cached_blocks":2178,"cache_hit":0,"new_prefill":38252,"score_linear":35708.0,"score_lmetric":55659},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1210,"cache_hit":36352,"new_prefill":1900,"score_linear":-36352.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":110978,"ongoing_decode_tokens":110978,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1223,"cache_hit":0,"new_prefill":38252,"score_linear":110978.0,"score_lmetric":114756},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":106951,"ongoing_decode_tokens":80246,"pending_prefill_tokens":12369,"num_requests":2,"active_p_offloads":0,"cached_blocks":1047,"cache_hit":0,"new_prefill":38252,"score_linear":106951.0,"score_lmetric":101242}],"chosen_score_linear":-36352.0,"chosen_score_lmetric":0,"t_first_token":348778.873071439,"t_first_token_unix":1779779643.8870149,"t_done":348782.573869985,"t_done_unix":1779779647.5878134},{"request_id":"1355484:2:1359370:525","session_id":"1355484","input_length":19493,"t_proxy_recv":348733.373225084,"t_decision_unix":1779779598.3871665,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":19456,"affinity_cache_ratio":0.9981018827271328,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":19456,"estimated_new_tokens":37,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":122597,"ongoing_decode_tokens":0,"pending_prefill_tokens":121573,"num_requests":1,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":19456,"new_prefill":37,"score_linear":103141.0,"score_lmetric":121610},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":19493,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1467,"cache_hit":0,"new_prefill":19493,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":134962,"ongoing_decode_tokens":0,"pending_prefill_tokens":83762,"num_requests":2,"active_p_offloads":0,"cached_blocks":1294,"cache_hit":0,"new_prefill":19493,"score_linear":134962.0,"score_lmetric":206510},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":48193,"ongoing_decode_tokens":48193,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1930,"cache_hit":0,"new_prefill":19493,"score_linear":48193.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1130,"cache_hit":0,"new_prefill":19493,"score_linear":68259.0,"score_lmetric":19493},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":58758,"ongoing_decode_tokens":0,"pending_prefill_tokens":45958,"num_requests":1,"active_p_offloads":0,"cached_blocks":1058,"cache_hit":12800,"new_prefill":6693,"score_linear":45958.0,"score_lmetric":52651},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":101617,"ongoing_decode_tokens":101617,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":995,"cache_hit":12800,"new_prefill":6693,"score_linear":88817.0,"score_lmetric":13386}],"chosen_score_linear":103141.0,"chosen_score_lmetric":121610,"t_first_token":348772.895037301,"t_first_token_unix":1779779637.9089804,"t_done":348784.401756369,"t_done_unix":1779779649.4157},{"request_id":"1356885:1:1356885:514","session_id":"1356885","input_length":122597,"t_proxy_recv":348724.637693569,"t_decision_unix":1779779589.6516347,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":false,"cache_hit":1024,"estimated_new_tokens":121573,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":1024,"new_prefill":121573,"score_linear":-1024.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":62618,"ongoing_decode_tokens":62618,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":968,"cache_hit":1024,"new_prefill":121573,"score_linear":61594.0,"score_lmetric":121573},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":35308,"ongoing_decode_tokens":0,"pending_prefill_tokens":35308,"num_requests":1,"active_p_offloads":0,"cached_blocks":1398,"cache_hit":0,"new_prefill":122597,"score_linear":35308.0,"score_lmetric":157905},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":134962,"ongoing_decode_tokens":0,"pending_prefill_tokens":83762,"num_requests":2,"active_p_offloads":0,"cached_blocks":1294,"cache_hit":0,"new_prefill":122597,"score_linear":134962.0,"score_lmetric":412718},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":121673,"ongoing_decode_tokens":24503,"pending_prefill_tokens":3986,"num_requests":2,"active_p_offloads":0,"cached_blocks":1882,"cache_hit":6656,"new_prefill":115941,"score_linear":115017.0,"score_lmetric":239854},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1129,"cache_hit":0,"new_prefill":122597,"score_linear":68259.0,"score_lmetric":122597},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1045,"cache_hit":0,"new_prefill":122597,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":99460,"ongoing_decode_tokens":99460,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":994,"cache_hit":0,"new_prefill":122597,"score_linear":99460.0,"score_lmetric":245194}],"chosen_score_linear":-1024.0,"chosen_score_lmetric":0,"t_first_token":348772.893519312,"t_first_token_unix":1779779637.9074628,"t_done":348784.451791413,"t_done_unix":1779779649.4657347},{"request_id":"1357132:2:1364990:550","session_id":"1357132","input_length":14721,"t_proxy_recv":348752.953825139,"t_decision_unix":1779779617.9677663,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":13312,"affinity_cache_ratio":0.9042863935873922,"affinity_num_requests":1,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"cache_hit":13312,"estimated_new_tokens":1409,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":240128,"ongoing_decode_tokens":0,"pending_prefill_tokens":121856,"num_requests":3,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":14721,"score_linear":240128.0,"score_lmetric":409731},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":117888,"ongoing_decode_tokens":0,"pending_prefill_tokens":117888,"num_requests":1,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":13312,"new_prefill":1409,"score_linear":104576.0,"score_lmetric":119297},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":18076,"ongoing_decode_tokens":16057,"pending_prefill_tokens":2019,"num_requests":2,"active_p_offloads":0,"cached_blocks":1543,"cache_hit":0,"new_prefill":14721,"score_linear":18076.0,"score_lmetric":33480},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76969,"ongoing_decode_tokens":0,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1458,"cache_hit":0,"new_prefill":14721,"score_linear":76969.0,"score_lmetric":91690},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90778,"ongoing_decode_tokens":0,"pending_prefill_tokens":63130,"num_requests":1,"active_p_offloads":0,"cached_blocks":1948,"cache_hit":0,"new_prefill":14721,"score_linear":90778.0,"score_lmetric":77851},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1131,"cache_hit":0,"new_prefill":14721,"score_linear":68259.0,"score_lmetric":14721},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":72150,"ongoing_decode_tokens":12636,"pending_prefill_tokens":35450,"num_requests":2,"active_p_offloads":0,"cached_blocks":1150,"cache_hit":0,"new_prefill":14721,"score_linear":72150.0,"score_lmetric":100342},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":94143,"ongoing_decode_tokens":94143,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1011,"cache_hit":0,"new_prefill":14721,"score_linear":94143.0,"score_lmetric":29442}],"chosen_score_linear":104576.0,"chosen_score_lmetric":119297,"t_first_token":348782.686120302,"t_first_token_unix":1779779647.7000635,"t_done":348784.556198943,"t_done_unix":1779779649.5701423},{"request_id":"1357132:3:1366582:557","session_id":"1357132","input_length":14816,"t_proxy_recv":348784.560646879,"t_decision_unix":1779779649.5745883,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":14336,"affinity_cache_ratio":0.9676025917926566,"affinity_num_requests":1,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":14336,"estimated_new_tokens":480,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":182283,"ongoing_decode_tokens":98038,"pending_prefill_tokens":5909,"num_requests":3,"active_p_offloads":0,"cached_blocks":1381,"cache_hit":0,"new_prefill":14816,"score_linear":182283.0,"score_lmetric":62175},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":117888,"ongoing_decode_tokens":117888,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1007,"cache_hit":14336,"new_prefill":480,"score_linear":103552.0,"score_lmetric":480},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126259,"ongoing_decode_tokens":0,"pending_prefill_tokens":91443,"num_requests":2,"active_p_offloads":0,"cached_blocks":1708,"cache_hit":0,"new_prefill":14816,"score_linear":126259.0,"score_lmetric":212518},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1750,"cache_hit":0,"new_prefill":14816,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":182626,"ongoing_decode_tokens":0,"pending_prefill_tokens":164194,"num_requests":1,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":14816,"score_linear":182626.0,"score_lmetric":179010},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1214,"cache_hit":0,"new_prefill":14816,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":78680,"ongoing_decode_tokens":78680,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1246,"cache_hit":0,"new_prefill":14816,"score_linear":78680.0,"score_lmetric":29632},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":106951,"ongoing_decode_tokens":106951,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1047,"cache_hit":0,"new_prefill":14816,"score_linear":106951.0,"score_lmetric":29632}],"chosen_score_linear":103552.0,"chosen_score_lmetric":480,"t_first_token":348784.627402586,"t_first_token_unix":1779779649.6413457,"t_done":348784.951810578,"t_done_unix":1779779649.9657543},{"request_id":"1363943:5:1373637:609","session_id":"1363943","input_length":18676,"t_proxy_recv":348783.448946057,"t_decision_unix":1779779648.4628873,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":18432,"affinity_cache_ratio":0.9869351038766331,"affinity_num_requests":1,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":18432,"estimated_new_tokens":244,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":304659,"ongoing_decode_tokens":240128,"pending_prefill_tokens":5651,"num_requests":4,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":8704,"new_prefill":9972,"score_linear":295955.0,"score_lmetric":62492},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":132609,"ongoing_decode_tokens":132609,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":18676,"score_linear":132609.0,"score_lmetric":37352},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":79495,"ongoing_decode_tokens":0,"pending_prefill_tokens":79495,"num_requests":1,"active_p_offloads":0,"cached_blocks":1708,"cache_hit":0,"new_prefill":18676,"score_linear":79495.0,"score_lmetric":98171},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1750,"cache_hit":0,"new_prefill":18676,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":182626,"ongoing_decode_tokens":0,"pending_prefill_tokens":164194,"num_requests":1,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":4096,"new_prefill":14580,"score_linear":178530.0,"score_lmetric":178774},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1214,"cache_hit":0,"new_prefill":18676,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":60004,"ongoing_decode_tokens":60004,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1246,"cache_hit":18432,"new_prefill":244,"score_linear":41572.0,"score_lmetric":244},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":106951,"ongoing_decode_tokens":106951,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1047,"cache_hit":4096,"new_prefill":14580,"score_linear":102855.0,"score_lmetric":29160}],"chosen_score_linear":41572.0,"chosen_score_lmetric":244,"t_first_token":348783.536912529,"t_first_token_unix":1779779648.5508556,"t_done":348785.352061626,"t_done_unix":1779779650.3660052},{"request_id":"1357132:4:1367654:570","session_id":"1357132","input_length":14949,"t_proxy_recv":348784.956566678,"t_decision_unix":1779779649.9705079,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":14336,"affinity_cache_ratio":0.9589939126362967,"affinity_num_requests":1,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":14336,"estimated_new_tokens":613,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":182283,"ongoing_decode_tokens":98038,"pending_prefill_tokens":5909,"num_requests":3,"active_p_offloads":0,"cached_blocks":1381,"cache_hit":0,"new_prefill":14949,"score_linear":182283.0,"score_lmetric":62574},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":117888,"ongoing_decode_tokens":117888,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1007,"cache_hit":14336,"new_prefill":613,"score_linear":103552.0,"score_lmetric":613},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126259,"ongoing_decode_tokens":0,"pending_prefill_tokens":91443,"num_requests":2,"active_p_offloads":0,"cached_blocks":1708,"cache_hit":0,"new_prefill":14949,"score_linear":126259.0,"score_lmetric":212784},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1750,"cache_hit":0,"new_prefill":14949,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":182626,"ongoing_decode_tokens":0,"pending_prefill_tokens":164194,"num_requests":1,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":14949,"score_linear":182626.0,"score_lmetric":179143},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1214,"cache_hit":0,"new_prefill":14949,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":78680,"ongoing_decode_tokens":78680,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1246,"cache_hit":0,"new_prefill":14949,"score_linear":78680.0,"score_lmetric":29898},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":106951,"ongoing_decode_tokens":106951,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1047,"cache_hit":0,"new_prefill":14949,"score_linear":106951.0,"score_lmetric":29898}],"chosen_score_linear":103552.0,"chosen_score_lmetric":613,"t_first_token":348785.044195259,"t_first_token_unix":1779779650.0581396,"t_done":348785.451400651,"t_done_unix":1779779650.4653437},{"request_id":"1313181:4:1368020:573","session_id":"1313181","input_length":60004,"t_proxy_recv":348770.289345811,"t_decision_unix":1779779635.303287,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":42496,"affinity_cache_ratio":0.7082194520365309,"affinity_num_requests":1,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":42496,"estimated_new_tokens":17508,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":240128,"ongoing_decode_tokens":0,"pending_prefill_tokens":121856,"num_requests":3,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":24064,"new_prefill":35940,"score_linear":216064.0,"score_lmetric":473388},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":132609,"ongoing_decode_tokens":0,"pending_prefill_tokens":119297,"num_requests":2,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":60004,"score_linear":132609.0,"score_lmetric":358602},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1665,"cache_hit":0,"new_prefill":60004,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":72919,"ongoing_decode_tokens":0,"pending_prefill_tokens":72919,"num_requests":1,"active_p_offloads":0,"cached_blocks":1608,"cache_hit":0,"new_prefill":60004,"score_linear":72919.0,"score_lmetric":132923},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":57195,"ongoing_decode_tokens":49483,"pending_prefill_tokens":7712,"num_requests":1,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":0,"new_prefill":60004,"score_linear":57195.0,"score_lmetric":67716},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1154,"cache_hit":0,"new_prefill":60004,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":18160,"ongoing_decode_tokens":18160,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1223,"cache_hit":42496,"new_prefill":17508,"score_linear":-24336.0,"score_lmetric":17508},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1047,"cache_hit":12800,"new_prefill":47204,"score_linear":67446.0,"score_lmetric":47204}],"chosen_score_linear":-24336.0,"chosen_score_lmetric":17508,"t_first_token":348776.213103678,"t_first_token_unix":1779779641.22705,"t_done":348785.495075637,"t_done_unix":1779779650.5090191},{"request_id":"1357132:5:1368445:579","session_id":"1357132","input_length":15871,"t_proxy_recv":348785.456162167,"t_decision_unix":1779779650.4701037,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":14848,"affinity_cache_ratio":0.93554281393737,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":14848,"estimated_new_tokens":1023,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":182283,"ongoing_decode_tokens":98038,"pending_prefill_tokens":5909,"num_requests":3,"active_p_offloads":0,"cached_blocks":1381,"cache_hit":0,"new_prefill":15871,"score_linear":182283.0,"score_lmetric":65340},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":117888,"ongoing_decode_tokens":117888,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1008,"cache_hit":14848,"new_prefill":1023,"score_linear":103040.0,"score_lmetric":1023},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126259,"ongoing_decode_tokens":0,"pending_prefill_tokens":91443,"num_requests":2,"active_p_offloads":0,"cached_blocks":1708,"cache_hit":0,"new_prefill":15871,"score_linear":126259.0,"score_lmetric":214628},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1750,"cache_hit":0,"new_prefill":15871,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":182626,"ongoing_decode_tokens":0,"pending_prefill_tokens":164194,"num_requests":1,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":15871,"score_linear":182626.0,"score_lmetric":180065},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1214,"cache_hit":0,"new_prefill":15871,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":60004,"ongoing_decode_tokens":60004,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1246,"cache_hit":0,"new_prefill":15871,"score_linear":60004.0,"score_lmetric":15871},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":106951,"ongoing_decode_tokens":106951,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1047,"cache_hit":0,"new_prefill":15871,"score_linear":106951.0,"score_lmetric":31742}],"chosen_score_linear":103040.0,"chosen_score_lmetric":1023,"t_first_token":348785.762751429,"t_first_token_unix":1779779650.7766945,"t_done":348786.152783249,"t_done_unix":1779779651.166726},{"request_id":"1374347:1:1374347:611","session_id":"1374347","input_length":19862,"t_proxy_recv":348785.889877848,"t_decision_unix":1779779650.9038193,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":19862,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":182283,"ongoing_decode_tokens":98038,"pending_prefill_tokens":5909,"num_requests":3,"active_p_offloads":0,"cached_blocks":1381,"cache_hit":0,"new_prefill":19862,"score_linear":182283.0,"score_lmetric":77313},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":133759,"ongoing_decode_tokens":133759,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1008,"cache_hit":0,"new_prefill":19862,"score_linear":133759.0,"score_lmetric":39724},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126259,"ongoing_decode_tokens":0,"pending_prefill_tokens":91443,"num_requests":2,"active_p_offloads":0,"cached_blocks":1708,"cache_hit":0,"new_prefill":19862,"score_linear":126259.0,"score_lmetric":222610},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1750,"cache_hit":0,"new_prefill":19862,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":182626,"ongoing_decode_tokens":0,"pending_prefill_tokens":164194,"num_requests":1,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":19862,"score_linear":182626.0,"score_lmetric":184056},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1214,"cache_hit":0,"new_prefill":19862,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":105902,"ongoing_decode_tokens":0,"pending_prefill_tokens":45486,"num_requests":1,"active_p_offloads":0,"cached_blocks":1249,"cache_hit":0,"new_prefill":19862,"score_linear":105902.0,"score_lmetric":65348},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":106951,"ongoing_decode_tokens":106951,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1047,"cache_hit":0,"new_prefill":19862,"score_linear":106951.0,"score_lmetric":39724}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348787.930733692,"t_first_token_unix":1779779652.9446774,"t_done":348788.307314519,"t_done_unix":1779779653.3212578},{"request_id":"1374465:1:1374465:613","session_id":"1374465","input_length":18497,"t_proxy_recv":348786.332199746,"t_decision_unix":1779779651.346141,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.5,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":18497,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":182283,"ongoing_decode_tokens":98038,"pending_prefill_tokens":5909,"num_requests":3,"active_p_offloads":0,"cached_blocks":1381,"cache_hit":0,"new_prefill":18497,"score_linear":182283.0,"score_lmetric":73218},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":117888,"ongoing_decode_tokens":117888,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1010,"cache_hit":0,"new_prefill":18497,"score_linear":117888.0,"score_lmetric":18497},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126259,"ongoing_decode_tokens":0,"pending_prefill_tokens":91443,"num_requests":2,"active_p_offloads":0,"cached_blocks":1708,"cache_hit":0,"new_prefill":18497,"score_linear":126259.0,"score_lmetric":219880},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1750,"cache_hit":0,"new_prefill":18497,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":246162,"ongoing_decode_tokens":0,"pending_prefill_tokens":164754,"num_requests":2,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":18497,"score_linear":246162.0,"score_lmetric":366502},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":19862,"ongoing_decode_tokens":0,"pending_prefill_tokens":19862,"num_requests":1,"active_p_offloads":0,"cached_blocks":1214,"cache_hit":0,"new_prefill":18497,"score_linear":19862.0,"score_lmetric":38359},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":105902,"ongoing_decode_tokens":0,"pending_prefill_tokens":45486,"num_requests":1,"active_p_offloads":0,"cached_blocks":1249,"cache_hit":0,"new_prefill":18497,"score_linear":105902.0,"score_lmetric":63983},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":106951,"ongoing_decode_tokens":106951,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1047,"cache_hit":0,"new_prefill":18497,"score_linear":106951.0,"score_lmetric":36994}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348788.16274928,"t_first_token_unix":1779779653.1766927,"t_done":348788.372350522,"t_done_unix":1779779653.3862934},{"request_id":"1364090:2:1371798:600","session_id":"1364090","input_length":26705,"t_proxy_recv":348776.909533966,"t_decision_unix":1779779641.9234753,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":14336,"affinity_cache_ratio":0.5368283093053735,"affinity_num_requests":1,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"cache_hit":14336,"estimated_new_tokens":12369,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":304659,"ongoing_decode_tokens":240128,"pending_prefill_tokens":5651,"num_requests":4,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":7680,"new_prefill":19025,"score_linear":296979.0,"score_lmetric":98704},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":132609,"ongoing_decode_tokens":0,"pending_prefill_tokens":119297,"num_requests":2,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":26705,"score_linear":132609.0,"score_lmetric":292004},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":35037,"ongoing_decode_tokens":0,"pending_prefill_tokens":11997,"num_requests":1,"active_p_offloads":0,"cached_blocks":1684,"cache_hit":0,"new_prefill":26705,"score_linear":35037.0,"score_lmetric":38702},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":72919,"ongoing_decode_tokens":0,"pending_prefill_tokens":72919,"num_requests":1,"active_p_offloads":0,"cached_blocks":1608,"cache_hit":0,"new_prefill":26705,"score_linear":72919.0,"score_lmetric":99624},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":18301,"ongoing_decode_tokens":18301,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2178,"cache_hit":7680,"new_prefill":19025,"score_linear":10621.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":13616,"ongoing_decode_tokens":0,"pending_prefill_tokens":13616,"num_requests":1,"active_p_offloads":0,"cached_blocks":1184,"cache_hit":0,"new_prefill":26705,"score_linear":13616.0,"score_lmetric":40321},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":110978,"ongoing_decode_tokens":78164,"pending_prefill_tokens":6190,"num_requests":3,"active_p_offloads":0,"cached_blocks":1223,"cache_hit":7680,"new_prefill":19025,"score_linear":103298.0,"score_lmetric":75645},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1047,"cache_hit":14336,"new_prefill":12369,"score_linear":65910.0,"score_lmetric":12369}],"chosen_score_linear":65910.0,"chosen_score_lmetric":12369,"t_first_token":348778.946812898,"t_first_token_unix":1779779643.960756,"t_done":348789.922803343,"t_done_unix":1779779654.9367492},{"request_id":"1268861:7:1374579:614","session_id":"1268861","input_length":41508,"t_proxy_recv":348786.797879413,"t_decision_unix":1779779651.8118205,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":36864,"affinity_cache_ratio":0.8881179531656548,"affinity_num_requests":1,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"cache_hit":36864,"estimated_new_tokens":4644,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":182283,"ongoing_decode_tokens":98038,"pending_prefill_tokens":5909,"num_requests":3,"active_p_offloads":0,"cached_blocks":1381,"cache_hit":0,"new_prefill":41508,"score_linear":182283.0,"score_lmetric":142251},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":117888,"ongoing_decode_tokens":117888,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1010,"cache_hit":36864,"new_prefill":4644,"score_linear":81024.0,"score_lmetric":4644},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126259,"ongoing_decode_tokens":0,"pending_prefill_tokens":91443,"num_requests":2,"active_p_offloads":0,"cached_blocks":1708,"cache_hit":0,"new_prefill":41508,"score_linear":126259.0,"score_lmetric":265902},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":18497,"ongoing_decode_tokens":0,"pending_prefill_tokens":18497,"num_requests":1,"active_p_offloads":0,"cached_blocks":1750,"cache_hit":0,"new_prefill":41508,"score_linear":18497.0,"score_lmetric":60005},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":246162,"ongoing_decode_tokens":0,"pending_prefill_tokens":164754,"num_requests":2,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":41508,"score_linear":246162.0,"score_lmetric":412524},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":19862,"ongoing_decode_tokens":0,"pending_prefill_tokens":19862,"num_requests":1,"active_p_offloads":0,"cached_blocks":1214,"cache_hit":0,"new_prefill":41508,"score_linear":19862.0,"score_lmetric":61370},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":105902,"ongoing_decode_tokens":0,"pending_prefill_tokens":45486,"num_requests":1,"active_p_offloads":0,"cached_blocks":1249,"cache_hit":0,"new_prefill":41508,"score_linear":105902.0,"score_lmetric":86994},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":106951,"ongoing_decode_tokens":106951,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1047,"cache_hit":12288,"new_prefill":29220,"score_linear":94663.0,"score_lmetric":58440}],"chosen_score_linear":81024.0,"chosen_score_lmetric":4644,"t_first_token":348788.002466538,"t_first_token_unix":1779779653.0164099,"t_done":348790.868128718,"t_done_unix":1779779655.882072},{"request_id":"1269373:22:1358956:522","session_id":"1269373","input_length":80246,"t_proxy_recv":348731.790755643,"t_decision_unix":1779779596.8046963,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":78336,"affinity_cache_ratio":0.9761981905640157,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":78336,"estimated_new_tokens":1910,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":122597,"ongoing_decode_tokens":0,"pending_prefill_tokens":121573,"num_requests":1,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":7680,"new_prefill":72566,"score_linear":114917.0,"score_lmetric":194139},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":80246,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1467,"cache_hit":0,"new_prefill":80246,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":134962,"ongoing_decode_tokens":0,"pending_prefill_tokens":83762,"num_requests":2,"active_p_offloads":0,"cached_blocks":1294,"cache_hit":0,"new_prefill":80246,"score_linear":134962.0,"score_lmetric":328016},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":124975,"ongoing_decode_tokens":124975,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1929,"cache_hit":7680,"new_prefill":72566,"score_linear":117295.0,"score_lmetric":145132},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1130,"cache_hit":0,"new_prefill":80246,"score_linear":68259.0,"score_lmetric":80246},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":58758,"ongoing_decode_tokens":0,"pending_prefill_tokens":45958,"num_requests":1,"active_p_offloads":0,"cached_blocks":1058,"cache_hit":7680,"new_prefill":72566,"score_linear":51078.0,"score_lmetric":118524},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":21371,"ongoing_decode_tokens":21371,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":995,"cache_hit":78336,"new_prefill":1910,"score_linear":-56965.0,"score_lmetric":1910}],"chosen_score_linear":-56965.0,"chosen_score_lmetric":1910,"t_first_token":348732.802103372,"t_first_token_unix":1779779597.8160474,"t_done":348791.877157261,"t_done_unix":1779779656.8911006},{"request_id":"1360649:1:1360649:527","session_id":"1360649","input_length":117888,"t_proxy_recv":348737.905358593,"t_decision_unix":1779779602.9192972,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":117888,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":142090,"ongoing_decode_tokens":0,"pending_prefill_tokens":121610,"num_requests":2,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":117888,"score_linear":142090.0,"score_lmetric":478996},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":117888,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1467,"cache_hit":0,"new_prefill":117888,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":134962,"ongoing_decode_tokens":0,"pending_prefill_tokens":83762,"num_requests":2,"active_p_offloads":0,"cached_blocks":1294,"cache_hit":0,"new_prefill":117888,"score_linear":134962.0,"score_lmetric":403300},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":48193,"ongoing_decode_tokens":48193,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1930,"cache_hit":0,"new_prefill":117888,"score_linear":48193.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1130,"cache_hit":0,"new_prefill":117888,"score_linear":68259.0,"score_lmetric":117888},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":58758,"ongoing_decode_tokens":0,"pending_prefill_tokens":45958,"num_requests":1,"active_p_offloads":0,"cached_blocks":1058,"cache_hit":0,"new_prefill":117888,"score_linear":58758.0,"score_lmetric":163846},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":101617,"ongoing_decode_tokens":101617,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":995,"cache_hit":0,"new_prefill":117888,"score_linear":101617.0,"score_lmetric":235776}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348782.684120018,"t_first_token_unix":1779779647.6980634,"t_done":348791.986479286,"t_done_unix":1779779657.0004225},{"request_id":"1374465:2:1376358:620","session_id":"1374465","input_length":21588,"t_proxy_recv":348792.734941697,"t_decision_unix":1779779657.748883,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":18432,"affinity_cache_ratio":0.8538076709282935,"affinity_num_requests":0,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":18432,"estimated_new_tokens":3156,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":218171,"ongoing_decode_tokens":182283,"pending_prefill_tokens":48,"num_requests":4,"active_p_offloads":0,"cached_blocks":1381,"cache_hit":0,"new_prefill":21588,"score_linear":218171.0,"score_lmetric":86544},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":119052,"ongoing_decode_tokens":119052,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1250,"cache_hit":0,"new_prefill":21588,"score_linear":119052.0,"score_lmetric":21588},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":164300,"ongoing_decode_tokens":0,"pending_prefill_tokens":92620,"num_requests":3,"active_p_offloads":0,"cached_blocks":1708,"cache_hit":0,"new_prefill":21588,"score_linear":164300.0,"score_lmetric":342624},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1786,"cache_hit":18432,"new_prefill":3156,"score_linear":-18432.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":246162,"ongoing_decode_tokens":0,"pending_prefill_tokens":164754,"num_requests":2,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":21588,"score_linear":246162.0,"score_lmetric":372684},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1252,"cache_hit":0,"new_prefill":21588,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":125018,"ongoing_decode_tokens":0,"pending_prefill_tokens":46170,"num_requests":2,"active_p_offloads":0,"cached_blocks":1249,"cache_hit":0,"new_prefill":21588,"score_linear":125018.0,"score_lmetric":135516},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":21588,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-18432.0,"chosen_score_lmetric":0,"t_first_token":348793.272324939,"t_first_token_unix":1779779658.2862685,"t_done":348793.586513208,"t_done_unix":1779779658.6004567},{"request_id":"1360649:2:1367022:561","session_id":"1360649","input_length":119052,"t_proxy_recv":348792.011593094,"t_decision_unix":1779779657.0255346,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":118272,"affinity_cache_ratio":0.9934482411047273,"affinity_num_requests":0,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":118272,"estimated_new_tokens":780,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":218171,"ongoing_decode_tokens":182283,"pending_prefill_tokens":48,"num_requests":4,"active_p_offloads":0,"cached_blocks":1381,"cache_hit":0,"new_prefill":119052,"score_linear":218171.0,"score_lmetric":476400},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1250,"cache_hit":118272,"new_prefill":780,"score_linear":-118272.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":164300,"ongoing_decode_tokens":0,"pending_prefill_tokens":92620,"num_requests":3,"active_p_offloads":0,"cached_blocks":1708,"cache_hit":0,"new_prefill":119052,"score_linear":164300.0,"score_lmetric":635016},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1786,"cache_hit":4096,"new_prefill":114956,"score_linear":-4096.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":246162,"ongoing_decode_tokens":0,"pending_prefill_tokens":164754,"num_requests":2,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":119052,"score_linear":246162.0,"score_lmetric":567612},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1252,"cache_hit":0,"new_prefill":119052,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":125018,"ongoing_decode_tokens":0,"pending_prefill_tokens":46170,"num_requests":2,"active_p_offloads":0,"cached_blocks":1249,"cache_hit":0,"new_prefill":119052,"score_linear":125018.0,"score_lmetric":330444},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":119052,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-118272.0,"chosen_score_lmetric":0,"t_first_token":348792.592586212,"t_first_token_unix":1779779657.6065297,"t_done":348793.908433932,"t_done_unix":1779779658.9223776},{"request_id":"1279412:24:1364169:546","session_id":"1279412","input_length":98038,"t_proxy_recv":348750.094640289,"t_decision_unix":1779779615.1085818,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":97792,"affinity_cache_ratio":0.9974907688855342,"affinity_num_requests":2,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":97792,"estimated_new_tokens":246,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":142090,"ongoing_decode_tokens":0,"pending_prefill_tokens":121610,"num_requests":2,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":97792,"new_prefill":246,"score_linear":44298.0,"score_lmetric":243712},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":117888,"ongoing_decode_tokens":0,"pending_prefill_tokens":117888,"num_requests":1,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":98038,"score_linear":117888.0,"score_lmetric":215926},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":24763,"ongoing_decode_tokens":16057,"pending_prefill_tokens":2050,"num_requests":2,"active_p_offloads":0,"cached_blocks":1539,"cache_hit":0,"new_prefill":98038,"score_linear":24763.0,"score_lmetric":200176},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76969,"ongoing_decode_tokens":0,"pending_prefill_tokens":76969,"num_requests":1,"active_p_offloads":0,"cached_blocks":1458,"cache_hit":0,"new_prefill":98038,"score_linear":76969.0,"score_lmetric":175007},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":90778,"ongoing_decode_tokens":0,"pending_prefill_tokens":63130,"num_requests":1,"active_p_offloads":0,"cached_blocks":1948,"cache_hit":0,"new_prefill":98038,"score_linear":90778.0,"score_lmetric":161168},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":68259,"ongoing_decode_tokens":68259,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1131,"cache_hit":0,"new_prefill":98038,"score_linear":68259.0,"score_lmetric":98038},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":12636,"ongoing_decode_tokens":0,"pending_prefill_tokens":8540,"num_requests":1,"active_p_offloads":0,"cached_blocks":1150,"cache_hit":12800,"new_prefill":85238,"score_linear":-164.0,"score_lmetric":93778},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":94143,"ongoing_decode_tokens":80246,"pending_prefill_tokens":2633,"num_requests":2,"active_p_offloads":0,"cached_blocks":1011,"cache_hit":12800,"new_prefill":85238,"score_linear":81343.0,"score_lmetric":175742}],"chosen_score_linear":44298.0,"chosen_score_lmetric":243712,"t_first_token":348772.898660008,"t_first_token_unix":1779779637.912603,"t_done":348794.663600791,"t_done_unix":1779779659.6775444},{"request_id":"1371337:2:1376534:621","session_id":"1371337","input_length":24323,"t_proxy_recv":348793.400636856,"t_decision_unix":1779779658.414578,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":4,"chosen_idx":5,"affinity_cache_hit":9216,"affinity_cache_ratio":0.3789006290342474,"affinity_num_requests":2,"avg_num_requests":1.625,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":24323,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":218171,"ongoing_decode_tokens":182283,"pending_prefill_tokens":48,"num_requests":4,"active_p_offloads":0,"cached_blocks":1381,"cache_hit":0,"new_prefill":24323,"score_linear":218171.0,"score_lmetric":97484},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":119052,"ongoing_decode_tokens":119052,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1250,"cache_hit":0,"new_prefill":24323,"score_linear":119052.0,"score_lmetric":24323},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":164300,"ongoing_decode_tokens":0,"pending_prefill_tokens":92620,"num_requests":3,"active_p_offloads":0,"cached_blocks":1708,"cache_hit":0,"new_prefill":24323,"score_linear":164300.0,"score_lmetric":350829},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":21588,"ongoing_decode_tokens":21588,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1786,"cache_hit":0,"new_prefill":24323,"score_linear":21588.0,"score_lmetric":24323},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":246162,"ongoing_decode_tokens":0,"pending_prefill_tokens":164754,"num_requests":2,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":9216,"new_prefill":15107,"score_linear":236946.0,"score_lmetric":359722},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1252,"cache_hit":0,"new_prefill":24323,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":125018,"ongoing_decode_tokens":0,"pending_prefill_tokens":46170,"num_requests":2,"active_p_offloads":0,"cached_blocks":1249,"cache_hit":0,"new_prefill":24323,"score_linear":125018.0,"score_lmetric":140986},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":24323,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348796.158585026,"t_first_token_unix":1779779661.172529,"t_done":348796.754601711,"t_done_unix":1779779661.7685454},{"request_id":"1377061:1:1377061:622","session_id":"1377061","input_length":16241,"t_proxy_recv":348795.367801816,"t_decision_unix":1779779660.381743,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.5,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":16241,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":120133,"ongoing_decode_tokens":84245,"pending_prefill_tokens":48,"num_requests":3,"active_p_offloads":0,"cached_blocks":1381,"cache_hit":0,"new_prefill":16241,"score_linear":120133.0,"score_lmetric":48867},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1251,"cache_hit":0,"new_prefill":16241,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":164300,"ongoing_decode_tokens":0,"pending_prefill_tokens":92620,"num_requests":3,"active_p_offloads":0,"cached_blocks":1708,"cache_hit":0,"new_prefill":16241,"score_linear":164300.0,"score_lmetric":326583},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1792,"cache_hit":0,"new_prefill":16241,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":246162,"ongoing_decode_tokens":0,"pending_prefill_tokens":164754,"num_requests":2,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":16241,"score_linear":246162.0,"score_lmetric":361990},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":24323,"ongoing_decode_tokens":0,"pending_prefill_tokens":24323,"num_requests":1,"active_p_offloads":0,"cached_blocks":1252,"cache_hit":0,"new_prefill":16241,"score_linear":24323.0,"score_lmetric":40564},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":125018,"ongoing_decode_tokens":0,"pending_prefill_tokens":46170,"num_requests":2,"active_p_offloads":0,"cached_blocks":1249,"cache_hit":0,"new_prefill":16241,"score_linear":125018.0,"score_lmetric":124822},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":98267,"ongoing_decode_tokens":0,"pending_prefill_tokens":85467,"num_requests":1,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":16241,"score_linear":98267.0,"score_lmetric":101708}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348796.852163387,"t_first_token_unix":1779779661.8661072,"t_done":348796.998281596,"t_done_unix":1779779662.0122247},{"request_id":"1355484:3:1361065:531","session_id":"1355484","input_length":19714,"t_proxy_recv":348784.406057662,"t_decision_unix":1779779649.4199991,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":19456,"affinity_cache_ratio":0.9869128538094755,"affinity_num_requests":3,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":19456,"estimated_new_tokens":258,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":285166,"ongoing_decode_tokens":220635,"pending_prefill_tokens":5651,"num_requests":3,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":19456,"new_prefill":258,"score_linear":265710.0,"score_lmetric":17727},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":132609,"ongoing_decode_tokens":132609,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":19714,"score_linear":132609.0,"score_lmetric":39428},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126259,"ongoing_decode_tokens":0,"pending_prefill_tokens":91443,"num_requests":2,"active_p_offloads":0,"cached_blocks":1708,"cache_hit":0,"new_prefill":19714,"score_linear":126259.0,"score_lmetric":222314},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1750,"cache_hit":0,"new_prefill":19714,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":182626,"ongoing_decode_tokens":0,"pending_prefill_tokens":164194,"num_requests":1,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":19714,"score_linear":182626.0,"score_lmetric":183908},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1214,"cache_hit":0,"new_prefill":19714,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":78680,"ongoing_decode_tokens":78680,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1246,"cache_hit":12800,"new_prefill":6914,"score_linear":65880.0,"score_lmetric":13828},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":106951,"ongoing_decode_tokens":106951,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1047,"cache_hit":12800,"new_prefill":6914,"score_linear":94151.0,"score_lmetric":13828}],"chosen_score_linear":265710.0,"chosen_score_lmetric":17727,"t_first_token":348791.864590983,"t_first_token_unix":1779779656.8785343,"t_done":348797.954707941,"t_done_unix":1779779662.9686513},{"request_id":"1342634:2:1370263:590","session_id":"1342634","input_length":64531,"t_proxy_recv":348771.642504759,"t_decision_unix":1779779636.6564462,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":58880,"affinity_cache_ratio":0.9124296849576172,"affinity_num_requests":3,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"cache_hit":58880,"estimated_new_tokens":5651,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":240128,"ongoing_decode_tokens":0,"pending_prefill_tokens":121856,"num_requests":3,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":58880,"new_prefill":5651,"score_linear":181248.0,"score_lmetric":382521},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":132609,"ongoing_decode_tokens":0,"pending_prefill_tokens":119297,"num_requests":2,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":64531,"score_linear":132609.0,"score_lmetric":367656},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":23307,"ongoing_decode_tokens":23307,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1665,"cache_hit":0,"new_prefill":64531,"score_linear":23307.0,"score_lmetric":64531},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":72919,"ongoing_decode_tokens":0,"pending_prefill_tokens":72919,"num_requests":1,"active_p_offloads":0,"cached_blocks":1608,"cache_hit":0,"new_prefill":64531,"score_linear":72919.0,"score_lmetric":137450},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":47802,"ongoing_decode_tokens":0,"pending_prefill_tokens":18618,"num_requests":1,"active_p_offloads":0,"cached_blocks":2145,"cache_hit":0,"new_prefill":64531,"score_linear":47802.0,"score_lmetric":83149},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":46546,"ongoing_decode_tokens":0,"pending_prefill_tokens":8146,"num_requests":2,"active_p_offloads":0,"cached_blocks":1154,"cache_hit":0,"new_prefill":64531,"score_linear":46546.0,"score_lmetric":145354},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":78164,"ongoing_decode_tokens":18160,"pending_prefill_tokens":17508,"num_requests":2,"active_p_offloads":0,"cached_blocks":1223,"cache_hit":0,"new_prefill":64531,"score_linear":78164.0,"score_lmetric":164078},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1047,"cache_hit":0,"new_prefill":64531,"score_linear":80246.0,"score_lmetric":64531}],"chosen_score_linear":181248.0,"chosen_score_lmetric":382521,"t_first_token":348791.863332746,"t_first_token_unix":1779779656.8772764,"t_done":348798.477010894,"t_done_unix":1779779663.4909554},{"request_id":"1372791:2:1377698:624","session_id":"1372791","input_length":14851,"t_proxy_recv":348797.602100449,"t_decision_unix":1779779662.6160414,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":4,"chosen_idx":3,"affinity_cache_hit":4608,"affinity_cache_ratio":0.3102821358831055,"affinity_num_requests":2,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":14851,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":120133,"ongoing_decode_tokens":120133,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1381,"cache_hit":0,"new_prefill":14851,"score_linear":120133.0,"score_lmetric":44553},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1251,"cache_hit":0,"new_prefill":14851,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":164300,"ongoing_decode_tokens":0,"pending_prefill_tokens":92620,"num_requests":3,"active_p_offloads":0,"cached_blocks":1708,"cache_hit":0,"new_prefill":14851,"score_linear":164300.0,"score_lmetric":322413},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1823,"cache_hit":0,"new_prefill":14851,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":246162,"ongoing_decode_tokens":0,"pending_prefill_tokens":164754,"num_requests":2,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":4608,"new_prefill":10243,"score_linear":241554.0,"score_lmetric":349994},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1299,"cache_hit":0,"new_prefill":14851,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":125018,"ongoing_decode_tokens":0,"pending_prefill_tokens":46170,"num_requests":2,"active_p_offloads":0,"cached_blocks":1249,"cache_hit":0,"new_prefill":14851,"score_linear":125018.0,"score_lmetric":122042},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":98267,"ongoing_decode_tokens":0,"pending_prefill_tokens":85467,"num_requests":1,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":14851,"score_linear":98267.0,"score_lmetric":100318}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348798.913570356,"t_first_token_unix":1779779663.9275136,"t_done":348799.110582995,"t_done_unix":1779779664.1245258},{"request_id":"1355484:4:1364784:548","session_id":"1355484","input_length":19806,"t_proxy_recv":348797.959264131,"t_decision_unix":1779779662.9732058,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":19456,"affinity_cache_ratio":0.9823285872967787,"affinity_num_requests":2,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":19456,"estimated_new_tokens":350,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":100419,"ongoing_decode_tokens":100419,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1381,"cache_hit":19456,"new_prefill":350,"score_linear":80963.0,"score_lmetric":700},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1251,"cache_hit":0,"new_prefill":19806,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":164300,"ongoing_decode_tokens":0,"pending_prefill_tokens":92620,"num_requests":3,"active_p_offloads":0,"cached_blocks":1708,"cache_hit":0,"new_prefill":19806,"score_linear":164300.0,"score_lmetric":337278},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":14851,"ongoing_decode_tokens":0,"pending_prefill_tokens":14851,"num_requests":1,"active_p_offloads":0,"cached_blocks":1823,"cache_hit":0,"new_prefill":19806,"score_linear":14851.0,"score_lmetric":34657},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":246162,"ongoing_decode_tokens":0,"pending_prefill_tokens":164754,"num_requests":2,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":19806,"score_linear":246162.0,"score_lmetric":369120},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1299,"cache_hit":0,"new_prefill":19806,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":125018,"ongoing_decode_tokens":0,"pending_prefill_tokens":46170,"num_requests":2,"active_p_offloads":0,"cached_blocks":1249,"cache_hit":12800,"new_prefill":7006,"score_linear":112218.0,"score_lmetric":106352},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":98267,"ongoing_decode_tokens":0,"pending_prefill_tokens":85467,"num_requests":1,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":12800,"new_prefill":7006,"score_linear":85467.0,"score_lmetric":92473}],"chosen_score_linear":80963.0,"chosen_score_lmetric":700,"t_first_token":348798.032983091,"t_first_token_unix":1779779663.046926,"t_done":348799.787340877,"t_done_unix":1779779664.8012846},{"request_id":"1355484:5:1366753:560","session_id":"1355484","input_length":20026,"t_proxy_recv":348799.792868918,"t_decision_unix":1779779664.8068104,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":19968,"affinity_cache_ratio":0.997103765105363,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":19968,"estimated_new_tokens":58,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":35888,"ongoing_decode_tokens":35888,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1393,"cache_hit":19968,"new_prefill":58,"score_linear":15920.0,"score_lmetric":58},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1251,"cache_hit":0,"new_prefill":20026,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":164300,"ongoing_decode_tokens":0,"pending_prefill_tokens":92620,"num_requests":3,"active_p_offloads":0,"cached_blocks":1708,"cache_hit":0,"new_prefill":20026,"score_linear":164300.0,"score_lmetric":337938},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1852,"cache_hit":0,"new_prefill":20026,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":246162,"ongoing_decode_tokens":0,"pending_prefill_tokens":164754,"num_requests":2,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":20026,"score_linear":246162.0,"score_lmetric":369560},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":22132,"ongoing_decode_tokens":0,"pending_prefill_tokens":6772,"num_requests":1,"active_p_offloads":0,"cached_blocks":1299,"cache_hit":0,"new_prefill":20026,"score_linear":22132.0,"score_lmetric":26798},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":125018,"ongoing_decode_tokens":0,"pending_prefill_tokens":46170,"num_requests":2,"active_p_offloads":0,"cached_blocks":1249,"cache_hit":12800,"new_prefill":7226,"score_linear":112218.0,"score_lmetric":106792},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":98267,"ongoing_decode_tokens":0,"pending_prefill_tokens":85467,"num_requests":1,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":12800,"new_prefill":7226,"score_linear":85467.0,"score_lmetric":92693}],"chosen_score_linear":15920.0,"chosen_score_lmetric":58,"t_first_token":348799.848815761,"t_first_token_unix":1779779664.862759,"t_done":348800.344691496,"t_done_unix":1779779665.3586347},{"request_id":"1377061:2:1378590:631","session_id":"1377061","input_length":16515,"t_proxy_recv":348800.631454114,"t_decision_unix":1779779665.6453955,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":15872,"affinity_cache_ratio":0.961065697850439,"affinity_num_requests":1,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":15872,"estimated_new_tokens":643,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":56046,"ongoing_decode_tokens":56046,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1393,"cache_hit":0,"new_prefill":16515,"score_linear":56046.0,"score_lmetric":33030},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1251,"cache_hit":0,"new_prefill":16515,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":164300,"ongoing_decode_tokens":0,"pending_prefill_tokens":92620,"num_requests":3,"active_p_offloads":0,"cached_blocks":1708,"cache_hit":0,"new_prefill":16515,"score_linear":164300.0,"score_lmetric":327405},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":5458,"ongoing_decode_tokens":0,"pending_prefill_tokens":5458,"num_requests":1,"active_p_offloads":0,"cached_blocks":1852,"cache_hit":15872,"new_prefill":643,"score_linear":-10414.0,"score_lmetric":6101},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":246162,"ongoing_decode_tokens":0,"pending_prefill_tokens":164754,"num_requests":2,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":16515,"score_linear":246162.0,"score_lmetric":362538},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":22132,"ongoing_decode_tokens":22132,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1299,"cache_hit":0,"new_prefill":16515,"score_linear":22132.0,"score_lmetric":16515},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":125018,"ongoing_decode_tokens":0,"pending_prefill_tokens":46170,"num_requests":2,"active_p_offloads":0,"cached_blocks":1249,"cache_hit":0,"new_prefill":16515,"score_linear":125018.0,"score_lmetric":125370},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":98267,"ongoing_decode_tokens":0,"pending_prefill_tokens":85467,"num_requests":1,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":16515,"score_linear":98267.0,"score_lmetric":101982}],"chosen_score_linear":-10414.0,"chosen_score_lmetric":6101,"t_first_token":348800.893939785,"t_first_token_unix":1779779665.907883,"t_done":348800.980088518,"t_done_unix":1779779665.9940317},{"request_id":"1355484:6:1369697:583","session_id":"1355484","input_length":20158,"t_proxy_recv":348800.350304768,"t_decision_unix":1779779665.3642466,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":19968,"affinity_cache_ratio":0.9905744617521579,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":19968,"estimated_new_tokens":190,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":35888,"ongoing_decode_tokens":35888,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1393,"cache_hit":19968,"new_prefill":190,"score_linear":15920.0,"score_lmetric":190},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1251,"cache_hit":0,"new_prefill":20158,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":164300,"ongoing_decode_tokens":0,"pending_prefill_tokens":92620,"num_requests":3,"active_p_offloads":0,"cached_blocks":1708,"cache_hit":0,"new_prefill":20158,"score_linear":164300.0,"score_lmetric":338334},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1852,"cache_hit":0,"new_prefill":20158,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":246162,"ongoing_decode_tokens":0,"pending_prefill_tokens":164754,"num_requests":2,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":20158,"score_linear":246162.0,"score_lmetric":369824},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":22132,"ongoing_decode_tokens":22132,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1299,"cache_hit":0,"new_prefill":20158,"score_linear":22132.0,"score_lmetric":20158},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":125018,"ongoing_decode_tokens":0,"pending_prefill_tokens":46170,"num_requests":2,"active_p_offloads":0,"cached_blocks":1249,"cache_hit":12800,"new_prefill":7358,"score_linear":112218.0,"score_lmetric":107056},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":98267,"ongoing_decode_tokens":0,"pending_prefill_tokens":85467,"num_requests":1,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":12800,"new_prefill":7358,"score_linear":85467.0,"score_lmetric":92825}],"chosen_score_linear":15920.0,"chosen_score_lmetric":190,"t_first_token":348800.422930793,"t_first_token_unix":1779779665.436874,"t_done":348801.193342814,"t_done_unix":1779779666.2072864},{"request_id":"1378684:1:1378684:632","session_id":"1378684","input_length":7810,"t_proxy_recv":348800.969489158,"t_decision_unix":1779779665.9834304,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.625,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":7810,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":56046,"ongoing_decode_tokens":56046,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1393,"cache_hit":0,"new_prefill":7810,"score_linear":56046.0,"score_lmetric":15620},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1251,"cache_hit":0,"new_prefill":7810,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":164300,"ongoing_decode_tokens":0,"pending_prefill_tokens":92620,"num_requests":3,"active_p_offloads":0,"cached_blocks":1708,"cache_hit":0,"new_prefill":7810,"score_linear":164300.0,"score_lmetric":301290},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":21973,"ongoing_decode_tokens":21973,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1852,"cache_hit":0,"new_prefill":7810,"score_linear":21973.0,"score_lmetric":15620},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":246162,"ongoing_decode_tokens":0,"pending_prefill_tokens":164754,"num_requests":2,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":7810,"score_linear":246162.0,"score_lmetric":345128},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":22132,"ongoing_decode_tokens":22132,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1299,"cache_hit":0,"new_prefill":7810,"score_linear":22132.0,"score_lmetric":7810},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":125018,"ongoing_decode_tokens":0,"pending_prefill_tokens":46170,"num_requests":2,"active_p_offloads":0,"cached_blocks":1249,"cache_hit":0,"new_prefill":7810,"score_linear":125018.0,"score_lmetric":107960},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":98267,"ongoing_decode_tokens":0,"pending_prefill_tokens":85467,"num_requests":1,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":7810,"score_linear":98267.0,"score_lmetric":93277}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348801.508026354,"t_first_token_unix":1779779666.5219696,"t_done":348801.77879491,"t_done_unix":1779779666.792738},{"request_id":"1286804:4:1378156:627","session_id":"1286804","input_length":22132,"t_proxy_recv":348799.107504111,"t_decision_unix":1779779664.1214454,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":15360,"affinity_cache_ratio":0.694017711910356,"affinity_num_requests":0,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":15360,"estimated_new_tokens":6772,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55694,"ongoing_decode_tokens":55694,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1392,"cache_hit":0,"new_prefill":22132,"score_linear":55694.0,"score_lmetric":44264},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1251,"cache_hit":0,"new_prefill":22132,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":164300,"ongoing_decode_tokens":0,"pending_prefill_tokens":92620,"num_requests":3,"active_p_offloads":0,"cached_blocks":1708,"cache_hit":0,"new_prefill":22132,"score_linear":164300.0,"score_lmetric":344256},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":14851,"ongoing_decode_tokens":14851,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1823,"cache_hit":0,"new_prefill":22132,"score_linear":14851.0,"score_lmetric":22132},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":246162,"ongoing_decode_tokens":0,"pending_prefill_tokens":164754,"num_requests":2,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":22132,"score_linear":246162.0,"score_lmetric":373772},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1299,"cache_hit":15360,"new_prefill":6772,"score_linear":-15360.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":125018,"ongoing_decode_tokens":0,"pending_prefill_tokens":46170,"num_requests":2,"active_p_offloads":0,"cached_blocks":1249,"cache_hit":0,"new_prefill":22132,"score_linear":125018.0,"score_lmetric":136604},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":98267,"ongoing_decode_tokens":0,"pending_prefill_tokens":85467,"num_requests":1,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":22132,"score_linear":98267.0,"score_lmetric":107599}],"chosen_score_linear":-15360.0,"chosen_score_lmetric":0,"t_first_token":348800.10866118,"t_first_token_unix":1779779665.1226048,"t_done":348801.962397137,"t_done_unix":1779779666.976341},{"request_id":"1378543:1:1378543:630","session_id":"1378543","input_length":5458,"t_proxy_recv":348800.475825535,"t_decision_unix":1779779665.4897668,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":5458,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":56046,"ongoing_decode_tokens":56046,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1393,"cache_hit":0,"new_prefill":5458,"score_linear":56046.0,"score_lmetric":10916},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1251,"cache_hit":0,"new_prefill":5458,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":164300,"ongoing_decode_tokens":0,"pending_prefill_tokens":92620,"num_requests":3,"active_p_offloads":0,"cached_blocks":1708,"cache_hit":0,"new_prefill":5458,"score_linear":164300.0,"score_lmetric":294234},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1852,"cache_hit":0,"new_prefill":5458,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":246162,"ongoing_decode_tokens":0,"pending_prefill_tokens":164754,"num_requests":2,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":5458,"score_linear":246162.0,"score_lmetric":340424},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":22132,"ongoing_decode_tokens":22132,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1299,"cache_hit":0,"new_prefill":5458,"score_linear":22132.0,"score_lmetric":5458},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":125018,"ongoing_decode_tokens":0,"pending_prefill_tokens":46170,"num_requests":2,"active_p_offloads":0,"cached_blocks":1249,"cache_hit":0,"new_prefill":5458,"score_linear":125018.0,"score_lmetric":103256},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":98267,"ongoing_decode_tokens":0,"pending_prefill_tokens":85467,"num_requests":1,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":5458,"score_linear":98267.0,"score_lmetric":90925}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348800.828283394,"t_first_token_unix":1779779665.842227,"t_done":348801.983651345,"t_done_unix":1779779666.9975944},{"request_id":"1379185:1:1379185:634","session_id":"1379185","input_length":1422,"t_proxy_recv":348802.686979637,"t_decision_unix":1779779667.7009208,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":1422,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":58880,"ongoing_decode_tokens":58880,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1393,"cache_hit":0,"new_prefill":1422,"score_linear":58880.0,"score_lmetric":2844},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":15283,"ongoing_decode_tokens":0,"pending_prefill_tokens":15283,"num_requests":1,"active_p_offloads":0,"cached_blocks":1266,"cache_hit":0,"new_prefill":1422,"score_linear":15283.0,"score_lmetric":16705},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":164300,"ongoing_decode_tokens":0,"pending_prefill_tokens":92620,"num_requests":3,"active_p_offloads":0,"cached_blocks":1708,"cache_hit":0,"new_prefill":1422,"score_linear":164300.0,"score_lmetric":282126},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1864,"cache_hit":0,"new_prefill":1422,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":246162,"ongoing_decode_tokens":0,"pending_prefill_tokens":164754,"num_requests":2,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":1422,"score_linear":246162.0,"score_lmetric":332352},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1312,"cache_hit":0,"new_prefill":1422,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":125018,"ongoing_decode_tokens":0,"pending_prefill_tokens":46170,"num_requests":2,"active_p_offloads":0,"cached_blocks":1249,"cache_hit":0,"new_prefill":1422,"score_linear":125018.0,"score_lmetric":95184},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":98267,"ongoing_decode_tokens":0,"pending_prefill_tokens":85467,"num_requests":1,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":1422,"score_linear":98267.0,"score_lmetric":86889}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348802.788175289,"t_first_token_unix":1779779667.8021185,"t_done":348802.972566728,"t_done_unix":1779779667.9865105},{"request_id":"1355484:7:1371284:596","session_id":"1355484","input_length":22992,"t_proxy_recv":348801.199352788,"t_decision_unix":1779779666.2132943,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":19968,"affinity_cache_ratio":0.8684759916492694,"affinity_num_requests":1,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":19968,"estimated_new_tokens":3024,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":35888,"ongoing_decode_tokens":35888,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1393,"cache_hit":19968,"new_prefill":3024,"score_linear":15920.0,"score_lmetric":3024},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":7810,"ongoing_decode_tokens":0,"pending_prefill_tokens":7810,"num_requests":1,"active_p_offloads":0,"cached_blocks":1251,"cache_hit":0,"new_prefill":22992,"score_linear":7810.0,"score_lmetric":30802},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":164300,"ongoing_decode_tokens":0,"pending_prefill_tokens":92620,"num_requests":3,"active_p_offloads":0,"cached_blocks":1708,"cache_hit":0,"new_prefill":22992,"score_linear":164300.0,"score_lmetric":346836},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":5458,"ongoing_decode_tokens":5458,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1853,"cache_hit":0,"new_prefill":22992,"score_linear":5458.0,"score_lmetric":22992},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":246162,"ongoing_decode_tokens":0,"pending_prefill_tokens":164754,"num_requests":2,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":22992,"score_linear":246162.0,"score_lmetric":375492},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":22132,"ongoing_decode_tokens":22132,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1299,"cache_hit":0,"new_prefill":22992,"score_linear":22132.0,"score_lmetric":22992},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":125018,"ongoing_decode_tokens":0,"pending_prefill_tokens":46170,"num_requests":2,"active_p_offloads":0,"cached_blocks":1249,"cache_hit":12800,"new_prefill":10192,"score_linear":112218.0,"score_lmetric":112724},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":98267,"ongoing_decode_tokens":0,"pending_prefill_tokens":85467,"num_requests":1,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":12800,"new_prefill":10192,"score_linear":85467.0,"score_lmetric":95659}],"chosen_score_linear":15920.0,"chosen_score_lmetric":3024,"t_first_token":348801.710646989,"t_first_token_unix":1779779666.7245903,"t_done":348804.232746239,"t_done_unix":1779779669.2466898},{"request_id":"1379569:1:1379569:638","session_id":"1379569","input_length":2632,"t_proxy_recv":348803.916232874,"t_decision_unix":1779779668.9301739,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.5,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":2632,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":58880,"ongoing_decode_tokens":58880,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1393,"cache_hit":0,"new_prefill":2632,"score_linear":58880.0,"score_lmetric":5264},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1266,"cache_hit":0,"new_prefill":2632,"score_linear":15283.0,"score_lmetric":2632},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":164300,"ongoing_decode_tokens":0,"pending_prefill_tokens":92620,"num_requests":3,"active_p_offloads":0,"cached_blocks":1708,"cache_hit":0,"new_prefill":2632,"score_linear":164300.0,"score_lmetric":285756},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":57656,"ongoing_decode_tokens":0,"pending_prefill_tokens":1336,"num_requests":1,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":2632,"score_linear":57656.0,"score_lmetric":3968},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":246162,"ongoing_decode_tokens":0,"pending_prefill_tokens":164754,"num_requests":2,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":2632,"score_linear":246162.0,"score_lmetric":334772},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1312,"cache_hit":0,"new_prefill":2632,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":125018,"ongoing_decode_tokens":0,"pending_prefill_tokens":46170,"num_requests":2,"active_p_offloads":0,"cached_blocks":1249,"cache_hit":0,"new_prefill":2632,"score_linear":125018.0,"score_lmetric":97604},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":98267,"ongoing_decode_tokens":0,"pending_prefill_tokens":85467,"num_requests":1,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":2632,"score_linear":98267.0,"score_lmetric":88099}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348804.081228206,"t_first_token_unix":1779779669.0951722,"t_done":348804.521477005,"t_done_unix":1779779669.53542},{"request_id":"1355484:8:1377993:625","session_id":"1355484","input_length":23457,"t_proxy_recv":348804.238793281,"t_decision_unix":1779779669.2527347,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":23040,"affinity_cache_ratio":0.9822227906381891,"affinity_num_requests":1,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":23040,"estimated_new_tokens":417,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":35888,"ongoing_decode_tokens":35888,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1399,"cache_hit":23040,"new_prefill":417,"score_linear":12848.0,"score_lmetric":417},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1266,"cache_hit":0,"new_prefill":23457,"score_linear":15283.0,"score_lmetric":23457},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":164300,"ongoing_decode_tokens":0,"pending_prefill_tokens":92620,"num_requests":3,"active_p_offloads":0,"cached_blocks":1708,"cache_hit":0,"new_prefill":23457,"score_linear":164300.0,"score_lmetric":348231},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":57656,"ongoing_decode_tokens":0,"pending_prefill_tokens":1336,"num_requests":1,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":23457,"score_linear":57656.0,"score_lmetric":24793},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":246162,"ongoing_decode_tokens":0,"pending_prefill_tokens":164754,"num_requests":2,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":23457,"score_linear":246162.0,"score_lmetric":376422},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":2632,"ongoing_decode_tokens":2632,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1312,"cache_hit":0,"new_prefill":23457,"score_linear":2632.0,"score_lmetric":23457},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":125018,"ongoing_decode_tokens":0,"pending_prefill_tokens":46170,"num_requests":2,"active_p_offloads":0,"cached_blocks":1249,"cache_hit":12800,"new_prefill":10657,"score_linear":112218.0,"score_lmetric":113654},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":98267,"ongoing_decode_tokens":0,"pending_prefill_tokens":85467,"num_requests":1,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":12800,"new_prefill":10657,"score_linear":85467.0,"score_lmetric":96124}],"chosen_score_linear":12848.0,"chosen_score_lmetric":417,"t_first_token":348804.338458218,"t_first_token_unix":1779779669.3524032,"t_done":348806.360059342,"t_done_unix":1779779671.374003},{"request_id":"1355484:9:1379722:640","session_id":"1355484","input_length":23717,"t_proxy_recv":348806.364934762,"t_decision_unix":1779779671.3788762,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":23552,"affinity_cache_ratio":0.9930429649618417,"affinity_num_requests":1,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":23552,"estimated_new_tokens":165,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":35888,"ongoing_decode_tokens":35888,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1400,"cache_hit":23552,"new_prefill":165,"score_linear":12336.0,"score_lmetric":165},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1266,"cache_hit":0,"new_prefill":23717,"score_linear":15283.0,"score_lmetric":23717},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":164300,"ongoing_decode_tokens":79495,"pending_prefill_tokens":13125,"num_requests":3,"active_p_offloads":0,"cached_blocks":1708,"cache_hit":0,"new_prefill":23717,"score_linear":164300.0,"score_lmetric":110526},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":75024,"ongoing_decode_tokens":0,"pending_prefill_tokens":2320,"num_requests":2,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":23717,"score_linear":75024.0,"score_lmetric":52074},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":246162,"ongoing_decode_tokens":0,"pending_prefill_tokens":164754,"num_requests":2,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":23717,"score_linear":246162.0,"score_lmetric":376942},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1317,"cache_hit":0,"new_prefill":23717,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":125018,"ongoing_decode_tokens":0,"pending_prefill_tokens":46170,"num_requests":2,"active_p_offloads":0,"cached_blocks":1249,"cache_hit":12800,"new_prefill":10917,"score_linear":112218.0,"score_lmetric":114174},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":98267,"ongoing_decode_tokens":0,"pending_prefill_tokens":85467,"num_requests":1,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":12800,"new_prefill":10917,"score_linear":85467.0,"score_lmetric":96384}],"chosen_score_linear":12336.0,"chosen_score_lmetric":165,"t_first_token":348806.423847472,"t_first_token_unix":1779779671.4377906,"t_done":348807.575990215,"t_done_unix":1779779672.5899334},{"request_id":"1379569:2:1380473:645","session_id":"1379569","input_length":6767,"t_proxy_recv":348807.101539745,"t_decision_unix":1779779672.115481,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":2560,"affinity_cache_ratio":0.37830648736515443,"affinity_num_requests":0,"avg_num_requests":1.875,"fallback_score":0,"tie_break_used":false,"cache_hit":2560,"estimated_new_tokens":4207,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":59605,"ongoing_decode_tokens":59605,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1400,"cache_hit":0,"new_prefill":6767,"score_linear":59605.0,"score_lmetric":13534},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1266,"cache_hit":0,"new_prefill":6767,"score_linear":15283.0,"score_lmetric":6767},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":164300,"ongoing_decode_tokens":79495,"pending_prefill_tokens":13125,"num_requests":3,"active_p_offloads":0,"cached_blocks":1708,"cache_hit":0,"new_prefill":6767,"score_linear":164300.0,"score_lmetric":59676},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":111245,"ongoing_decode_tokens":0,"pending_prefill_tokens":11917,"num_requests":4,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":6767,"score_linear":111245.0,"score_lmetric":74736},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":246162,"ongoing_decode_tokens":0,"pending_prefill_tokens":164754,"num_requests":2,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":6767,"score_linear":246162.0,"score_lmetric":343042},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1317,"cache_hit":2560,"new_prefill":4207,"score_linear":-2560.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":125018,"ongoing_decode_tokens":0,"pending_prefill_tokens":46170,"num_requests":2,"active_p_offloads":0,"cached_blocks":1249,"cache_hit":0,"new_prefill":6767,"score_linear":125018.0,"score_lmetric":105874},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":98267,"ongoing_decode_tokens":0,"pending_prefill_tokens":85467,"num_requests":1,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":6767,"score_linear":98267.0,"score_lmetric":92234}],"chosen_score_linear":-2560.0,"chosen_score_lmetric":0,"t_first_token":348807.438549,"t_first_token_unix":1779779672.4524927,"t_done":348808.058147296,"t_done_unix":1779779673.0720904},{"request_id":"1366128:5:1373829:610","session_id":"1366128","input_length":46764,"t_proxy_recv":348784.104979762,"t_decision_unix":1779779649.1189208,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":34816,"affinity_cache_ratio":0.7445043195620563,"affinity_num_requests":1,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":34816,"estimated_new_tokens":11948,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":304659,"ongoing_decode_tokens":240128,"pending_prefill_tokens":5651,"num_requests":4,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":46764,"score_linear":304659.0,"score_lmetric":209660},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":132609,"ongoing_decode_tokens":132609,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":46764,"score_linear":132609.0,"score_lmetric":93528},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":79495,"ongoing_decode_tokens":0,"pending_prefill_tokens":79495,"num_requests":1,"active_p_offloads":0,"cached_blocks":1708,"cache_hit":34816,"new_prefill":11948,"score_linear":44679.0,"score_lmetric":91443},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1750,"cache_hit":0,"new_prefill":46764,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":182626,"ongoing_decode_tokens":0,"pending_prefill_tokens":164194,"num_requests":1,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":46764,"score_linear":182626.0,"score_lmetric":210958},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1214,"cache_hit":0,"new_prefill":46764,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":78680,"ongoing_decode_tokens":78680,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1246,"cache_hit":0,"new_prefill":46764,"score_linear":78680.0,"score_lmetric":93528},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":106951,"ongoing_decode_tokens":106951,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1047,"cache_hit":0,"new_prefill":46764,"score_linear":106951.0,"score_lmetric":93528}],"chosen_score_linear":44679.0,"chosen_score_lmetric":91443,"t_first_token":348808.072174411,"t_first_token_unix":1779779673.0861177,"t_done":348808.718820909,"t_done_unix":1779779673.7327647},{"request_id":"1363943:6:1375416:618","session_id":"1363943","input_length":19116,"t_proxy_recv":348789.641779826,"t_decision_unix":1779779654.6557212,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":18432,"affinity_cache_ratio":0.9642184557438794,"affinity_num_requests":1,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"cache_hit":18432,"estimated_new_tokens":684,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":218171,"ongoing_decode_tokens":98038,"pending_prefill_tokens":5957,"num_requests":4,"active_p_offloads":0,"cached_blocks":1381,"cache_hit":8704,"new_prefill":10412,"score_linear":209467.0,"score_lmetric":65476},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":159396,"ongoing_decode_tokens":159396,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1010,"cache_hit":0,"new_prefill":19116,"score_linear":159396.0,"score_lmetric":38232},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126259,"ongoing_decode_tokens":0,"pending_prefill_tokens":91443,"num_requests":2,"active_p_offloads":0,"cached_blocks":1708,"cache_hit":0,"new_prefill":19116,"score_linear":126259.0,"score_lmetric":221118},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1786,"cache_hit":0,"new_prefill":19116,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":246162,"ongoing_decode_tokens":0,"pending_prefill_tokens":164754,"num_requests":2,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":4096,"new_prefill":15020,"score_linear":242066.0,"score_lmetric":359548},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1252,"cache_hit":0,"new_prefill":19116,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":105902,"ongoing_decode_tokens":0,"pending_prefill_tokens":45486,"num_requests":1,"active_p_offloads":0,"cached_blocks":1249,"cache_hit":18432,"new_prefill":684,"score_linear":87470.0,"score_lmetric":46170},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":106951,"ongoing_decode_tokens":106951,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1047,"cache_hit":4096,"new_prefill":15020,"score_linear":102855.0,"score_lmetric":30040}],"chosen_score_linear":87470.0,"chosen_score_lmetric":46170,"t_first_token":348809.030926743,"t_first_token_unix":1779779674.04487,"t_done":348811.007018781,"t_done_unix":1779779676.0209618},{"request_id":"1381728:1:1381728:651","session_id":"1381728","input_length":1391,"t_proxy_recv":348811.633772145,"t_decision_unix":1779779676.647714,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.125,"fallback_score":1391,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":1391,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":35888,"ongoing_decode_tokens":35888,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1400,"cache_hit":0,"new_prefill":1391,"score_linear":35888.0,"score_lmetric":1391},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1266,"cache_hit":0,"new_prefill":1391,"score_linear":15283.0,"score_lmetric":1391},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":186692,"ongoing_decode_tokens":117536,"pending_prefill_tokens":12324,"num_requests":4,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":1391,"score_linear":186692.0,"score_lmetric":54860},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":111245,"ongoing_decode_tokens":0,"pending_prefill_tokens":11917,"num_requests":4,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":1391,"score_linear":111245.0,"score_lmetric":53232},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":246162,"ongoing_decode_tokens":0,"pending_prefill_tokens":164754,"num_requests":2,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":1391,"score_linear":246162.0,"score_lmetric":332290},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":21609,"ongoing_decode_tokens":21609,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1325,"cache_hit":0,"new_prefill":1391,"score_linear":21609.0,"score_lmetric":2782},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":126629,"ongoing_decode_tokens":126629,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1250,"cache_hit":0,"new_prefill":1391,"score_linear":126629.0,"score_lmetric":2782},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":98267,"ongoing_decode_tokens":0,"pending_prefill_tokens":85467,"num_requests":1,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":1391,"score_linear":98267.0,"score_lmetric":86858}],"chosen_score_linear":15283.0,"chosen_score_lmetric":1391,"t_first_token":348811.785356034,"t_first_token_unix":1779779676.7992992,"t_done":348811.988421014,"t_done_unix":1779779677.002364},{"request_id":"1379569:3:1381510:649","session_id":"1379569","input_length":7620,"t_proxy_recv":348810.730283252,"t_decision_unix":1779779675.744224,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":6656,"affinity_cache_ratio":0.873490813648294,"affinity_num_requests":1,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"cache_hit":6656,"estimated_new_tokens":964,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":35888,"ongoing_decode_tokens":35888,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1400,"cache_hit":0,"new_prefill":7620,"score_linear":35888.0,"score_lmetric":7620},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1266,"cache_hit":0,"new_prefill":7620,"score_linear":15283.0,"score_lmetric":7620},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":176047,"ongoing_decode_tokens":117536,"pending_prefill_tokens":11919,"num_requests":3,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":7620,"score_linear":176047.0,"score_lmetric":58617},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":111245,"ongoing_decode_tokens":0,"pending_prefill_tokens":11917,"num_requests":4,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":7620,"score_linear":111245.0,"score_lmetric":78148},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":246162,"ongoing_decode_tokens":0,"pending_prefill_tokens":164754,"num_requests":2,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":7620,"score_linear":246162.0,"score_lmetric":344748},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":13989,"ongoing_decode_tokens":0,"pending_prefill_tokens":13989,"num_requests":1,"active_p_offloads":0,"cached_blocks":1325,"cache_hit":6656,"new_prefill":964,"score_linear":7333.0,"score_lmetric":14953},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":125018,"ongoing_decode_tokens":125018,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1249,"cache_hit":0,"new_prefill":7620,"score_linear":125018.0,"score_lmetric":15240},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":98267,"ongoing_decode_tokens":0,"pending_prefill_tokens":85467,"num_requests":1,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":7620,"score_linear":98267.0,"score_lmetric":93087}],"chosen_score_linear":7333.0,"chosen_score_lmetric":14953,"t_first_token":348811.597858878,"t_first_token_unix":1779779676.6118019,"t_done":348812.265086017,"t_done_unix":1779779677.2790296},{"request_id":"1363943:7:1377305:623","session_id":"1363943","input_length":20727,"t_proxy_recv":348811.01555876,"t_decision_unix":1779779676.0295005,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":18944,"affinity_cache_ratio":0.9139769382930477,"affinity_num_requests":1,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"cache_hit":18944,"estimated_new_tokens":1783,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":35888,"ongoing_decode_tokens":35888,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1400,"cache_hit":8704,"new_prefill":12023,"score_linear":27184.0,"score_lmetric":12023},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1266,"cache_hit":0,"new_prefill":20727,"score_linear":15283.0,"score_lmetric":20727},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":176047,"ongoing_decode_tokens":117536,"pending_prefill_tokens":11919,"num_requests":3,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":20727,"score_linear":176047.0,"score_lmetric":97938},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":111245,"ongoing_decode_tokens":0,"pending_prefill_tokens":11917,"num_requests":4,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":20727,"score_linear":111245.0,"score_lmetric":130576},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":246162,"ongoing_decode_tokens":0,"pending_prefill_tokens":164754,"num_requests":2,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":4096,"new_prefill":16631,"score_linear":242066.0,"score_lmetric":362770},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":21609,"ongoing_decode_tokens":0,"pending_prefill_tokens":14953,"num_requests":2,"active_p_offloads":0,"cached_blocks":1325,"cache_hit":0,"new_prefill":20727,"score_linear":21609.0,"score_lmetric":71360},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":105902,"ongoing_decode_tokens":105902,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1250,"cache_hit":18944,"new_prefill":1783,"score_linear":86958.0,"score_lmetric":1783},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":98267,"ongoing_decode_tokens":0,"pending_prefill_tokens":85467,"num_requests":1,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":4096,"new_prefill":16631,"score_linear":94171.0,"score_lmetric":102098}],"chosen_score_linear":86958.0,"chosen_score_lmetric":1783,"t_first_token":348811.35451417,"t_first_token_unix":1779779676.3684578,"t_done":348813.731281214,"t_done_unix":1779779678.745225},{"request_id":"1355951:3:1375866:619","session_id":"1355951","input_length":38041,"t_proxy_recv":348791.276426958,"t_decision_unix":1779779656.2903683,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":36864,"affinity_cache_ratio":0.9690596987460898,"affinity_num_requests":2,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":36864,"estimated_new_tokens":1177,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":218171,"ongoing_decode_tokens":98038,"pending_prefill_tokens":5957,"num_requests":4,"active_p_offloads":0,"cached_blocks":1381,"cache_hit":0,"new_prefill":38041,"score_linear":218171.0,"score_lmetric":175992},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":117888,"ongoing_decode_tokens":117888,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1019,"cache_hit":0,"new_prefill":38041,"score_linear":117888.0,"score_lmetric":38041},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126259,"ongoing_decode_tokens":0,"pending_prefill_tokens":91443,"num_requests":2,"active_p_offloads":0,"cached_blocks":1708,"cache_hit":36864,"new_prefill":1177,"score_linear":89395.0,"score_lmetric":185240},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1786,"cache_hit":0,"new_prefill":38041,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":246162,"ongoing_decode_tokens":0,"pending_prefill_tokens":164754,"num_requests":2,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":38041,"score_linear":246162.0,"score_lmetric":405590},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1252,"cache_hit":0,"new_prefill":38041,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":125018,"ongoing_decode_tokens":0,"pending_prefill_tokens":46170,"num_requests":2,"active_p_offloads":0,"cached_blocks":1249,"cache_hit":0,"new_prefill":38041,"score_linear":125018.0,"score_lmetric":168422},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":80246,"ongoing_decode_tokens":80246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1073,"cache_hit":0,"new_prefill":38041,"score_linear":80246.0,"score_lmetric":38041}],"chosen_score_linear":89395.0,"chosen_score_lmetric":185240,"t_first_token":348808.07672733,"t_first_token_unix":1779779673.0906703,"t_done":348813.933214863,"t_done_unix":1779779678.947158},{"request_id":"1313181:5:1320817:349","session_id":"1313181","input_length":105902,"t_proxy_recv":348785.545140377,"t_decision_unix":1779779650.5590818,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":60416,"affinity_cache_ratio":0.5704896980227002,"affinity_num_requests":0,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":60416,"estimated_new_tokens":45486,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":182283,"ongoing_decode_tokens":98038,"pending_prefill_tokens":5909,"num_requests":3,"active_p_offloads":0,"cached_blocks":1381,"cache_hit":24064,"new_prefill":81838,"score_linear":158219.0,"score_lmetric":263241},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":133759,"ongoing_decode_tokens":117888,"pending_prefill_tokens":1023,"num_requests":2,"active_p_offloads":0,"cached_blocks":1008,"cache_hit":0,"new_prefill":105902,"score_linear":133759.0,"score_lmetric":213850},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126259,"ongoing_decode_tokens":0,"pending_prefill_tokens":91443,"num_requests":2,"active_p_offloads":0,"cached_blocks":1708,"cache_hit":0,"new_prefill":105902,"score_linear":126259.0,"score_lmetric":394690},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1750,"cache_hit":0,"new_prefill":105902,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":182626,"ongoing_decode_tokens":0,"pending_prefill_tokens":164194,"num_requests":1,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":105902,"score_linear":182626.0,"score_lmetric":270096},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1214,"cache_hit":0,"new_prefill":105902,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1249,"cache_hit":60416,"new_prefill":45486,"score_linear":-60416.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":106951,"ongoing_decode_tokens":106951,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1047,"cache_hit":12800,"new_prefill":93102,"score_linear":94151.0,"score_lmetric":186204}],"chosen_score_linear":-60416.0,"chosen_score_lmetric":0,"t_first_token":348809.029632398,"t_first_token_unix":1779779674.043576,"t_done":348813.983552461,"t_done_unix":1779779678.9974961},{"request_id":"1382294:1:1382294:656","session_id":"1382294","input_length":3034,"t_proxy_recv":348813.637414964,"t_decision_unix":1779779678.651356,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.375,"fallback_score":3034,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":3034,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":35888,"ongoing_decode_tokens":35888,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1400,"cache_hit":0,"new_prefill":3034,"score_linear":35888.0,"score_lmetric":3034},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1268,"cache_hit":0,"new_prefill":3034,"score_linear":15283.0,"score_lmetric":3034},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":224803,"ongoing_decode_tokens":186692,"pending_prefill_tokens":3295,"num_requests":5,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":3034,"score_linear":224803.0,"score_lmetric":31645},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":111245,"ongoing_decode_tokens":0,"pending_prefill_tokens":11917,"num_requests":4,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":3034,"score_linear":111245.0,"score_lmetric":59804},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":246162,"ongoing_decode_tokens":0,"pending_prefill_tokens":164754,"num_requests":2,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":3034,"score_linear":246162.0,"score_lmetric":335576},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":68001,"ongoing_decode_tokens":13989,"pending_prefill_tokens":54012,"num_requests":2,"active_p_offloads":0,"cached_blocks":1327,"cache_hit":0,"new_prefill":3034,"score_linear":68001.0,"score_lmetric":114092},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":126629,"ongoing_decode_tokens":126629,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1250,"cache_hit":0,"new_prefill":3034,"score_linear":126629.0,"score_lmetric":6068},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":145668,"ongoing_decode_tokens":0,"pending_prefill_tokens":105220,"num_requests":2,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":3034,"score_linear":145668.0,"score_lmetric":216508}],"chosen_score_linear":15283.0,"chosen_score_lmetric":3034,"t_first_token":348813.8291452,"t_first_token_unix":1779779678.8430884,"t_done":348814.046893439,"t_done_unix":1779779679.0608366},{"request_id":"1382089:1:1382089:655","session_id":"1382089","input_length":38111,"t_proxy_recv":348812.910010038,"t_decision_unix":1779779677.923951,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.125,"fallback_score":14800,"tie_break_used":false,"cache_hit":34816,"estimated_new_tokens":3295,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":35888,"ongoing_decode_tokens":35888,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1400,"cache_hit":0,"new_prefill":38111,"score_linear":35888.0,"score_lmetric":38111},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1268,"cache_hit":0,"new_prefill":38111,"score_linear":15283.0,"score_lmetric":38111},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":186692,"ongoing_decode_tokens":176047,"pending_prefill_tokens":405,"num_requests":4,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":34816,"new_prefill":3295,"score_linear":151876.0,"score_lmetric":14800},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":111245,"ongoing_decode_tokens":0,"pending_prefill_tokens":11917,"num_requests":4,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":38111,"score_linear":111245.0,"score_lmetric":200112},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":246162,"ongoing_decode_tokens":0,"pending_prefill_tokens":164754,"num_requests":2,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":38111,"score_linear":246162.0,"score_lmetric":405730},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":13989,"ongoing_decode_tokens":13989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1327,"cache_hit":0,"new_prefill":38111,"score_linear":13989.0,"score_lmetric":38111},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":126629,"ongoing_decode_tokens":126629,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1250,"cache_hit":0,"new_prefill":38111,"score_linear":126629.0,"score_lmetric":76222},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":145668,"ongoing_decode_tokens":0,"pending_prefill_tokens":105220,"num_requests":2,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":38111,"score_linear":145668.0,"score_lmetric":286662}],"chosen_score_linear":151876.0,"chosen_score_lmetric":14800,"t_first_token":348813.751896538,"t_first_token_unix":1779779678.76584,"t_done":348814.179694568,"t_done_unix":1779779679.1936378},{"request_id":"1362265:4:1381673:650","session_id":"1362265","input_length":10645,"t_proxy_recv":348811.41279444,"t_decision_unix":1779779676.4267359,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":10240,"affinity_cache_ratio":0.9619539689995303,"affinity_num_requests":3,"avg_num_requests":2.0,"fallback_score":null,"tie_break_used":false,"cache_hit":10240,"estimated_new_tokens":405,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":35888,"ongoing_decode_tokens":35888,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1400,"cache_hit":0,"new_prefill":10645,"score_linear":35888.0,"score_lmetric":10645},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1266,"cache_hit":0,"new_prefill":10645,"score_linear":15283.0,"score_lmetric":10645},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":176047,"ongoing_decode_tokens":117536,"pending_prefill_tokens":11919,"num_requests":3,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":10240,"new_prefill":405,"score_linear":165807.0,"score_lmetric":36972},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":111245,"ongoing_decode_tokens":0,"pending_prefill_tokens":11917,"num_requests":4,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":10645,"score_linear":111245.0,"score_lmetric":90248},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":246162,"ongoing_decode_tokens":0,"pending_prefill_tokens":164754,"num_requests":2,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":10645,"score_linear":246162.0,"score_lmetric":350798},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":21609,"ongoing_decode_tokens":0,"pending_prefill_tokens":14953,"num_requests":2,"active_p_offloads":0,"cached_blocks":1325,"cache_hit":0,"new_prefill":10645,"score_linear":21609.0,"score_lmetric":51196},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":126629,"ongoing_decode_tokens":126629,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1250,"cache_hit":0,"new_prefill":10645,"score_linear":126629.0,"score_lmetric":21290},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":98267,"ongoing_decode_tokens":0,"pending_prefill_tokens":85467,"num_requests":1,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":10645,"score_linear":98267.0,"score_lmetric":96112}],"chosen_score_linear":165807.0,"chosen_score_lmetric":36972,"t_first_token":348812.921316043,"t_first_token_unix":1779779677.9352589,"t_done":348814.375225335,"t_done_unix":1779779679.3891687},{"request_id":"1301929:2:1375154:617","session_id":"1301929","input_length":35888,"t_proxy_recv":348788.784352779,"t_decision_unix":1779779653.798294,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":35840,"affinity_cache_ratio":0.9986625055728935,"affinity_num_requests":3,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":35840,"estimated_new_tokens":48,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":182283,"ongoing_decode_tokens":98038,"pending_prefill_tokens":5909,"num_requests":3,"active_p_offloads":0,"cached_blocks":1381,"cache_hit":35840,"new_prefill":48,"score_linear":146443.0,"score_lmetric":17871},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":159396,"ongoing_decode_tokens":159396,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1010,"cache_hit":0,"new_prefill":35888,"score_linear":159396.0,"score_lmetric":71776},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126259,"ongoing_decode_tokens":0,"pending_prefill_tokens":91443,"num_requests":2,"active_p_offloads":0,"cached_blocks":1708,"cache_hit":0,"new_prefill":35888,"score_linear":126259.0,"score_lmetric":254662},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1786,"cache_hit":0,"new_prefill":35888,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":246162,"ongoing_decode_tokens":0,"pending_prefill_tokens":164754,"num_requests":2,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":35888,"score_linear":246162.0,"score_lmetric":401284},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1252,"cache_hit":0,"new_prefill":35888,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":105902,"ongoing_decode_tokens":0,"pending_prefill_tokens":45486,"num_requests":1,"active_p_offloads":0,"cached_blocks":1249,"cache_hit":0,"new_prefill":35888,"score_linear":105902.0,"score_lmetric":81374},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":106951,"ongoing_decode_tokens":106951,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1047,"cache_hit":0,"new_prefill":35888,"score_linear":106951.0,"score_lmetric":71776}],"chosen_score_linear":146443.0,"chosen_score_lmetric":17871,"t_first_token":348797.177725813,"t_first_token_unix":1779779662.19167,"t_done":348814.827823371,"t_done_unix":1779779679.8417664},{"request_id":"1382667:1:1382667:658","session_id":"1382667","input_length":1955,"t_proxy_recv":348815.055017354,"t_decision_unix":1779779680.0689406,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.875,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":1955,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1400,"cache_hit":0,"new_prefill":1955,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1273,"cache_hit":0,"new_prefill":1955,"score_linear":15283.0,"score_lmetric":1955},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":138006,"ongoing_decode_tokens":138006,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1736,"cache_hit":0,"new_prefill":1955,"score_linear":138006.0,"score_lmetric":3910},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":111245,"ongoing_decode_tokens":0,"pending_prefill_tokens":11917,"num_requests":4,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":1955,"score_linear":111245.0,"score_lmetric":55488},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":246162,"ongoing_decode_tokens":0,"pending_prefill_tokens":164754,"num_requests":2,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":1955,"score_linear":246162.0,"score_lmetric":333418},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":68001,"ongoing_decode_tokens":13989,"pending_prefill_tokens":54012,"num_requests":2,"active_p_offloads":0,"cached_blocks":1327,"cache_hit":0,"new_prefill":1955,"score_linear":68001.0,"score_lmetric":111934},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":81806,"ongoing_decode_tokens":81806,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1341,"cache_hit":0,"new_prefill":1955,"score_linear":81806.0,"score_lmetric":3910},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":145668,"ongoing_decode_tokens":0,"pending_prefill_tokens":105220,"num_requests":2,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":1955,"score_linear":145668.0,"score_lmetric":214350}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348815.183320534,"t_first_token_unix":1779779680.197264,"t_done":348815.464466932,"t_done_unix":1779779680.47841},{"request_id":"1382710:1:1382710:659","session_id":"1382710","input_length":3190,"t_proxy_recv":348815.153207961,"t_decision_unix":1779779680.1671493,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.0,"fallback_score":3190,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":3190,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":1955,"ongoing_decode_tokens":0,"pending_prefill_tokens":1955,"num_requests":1,"active_p_offloads":0,"cached_blocks":1400,"cache_hit":0,"new_prefill":3190,"score_linear":1955.0,"score_lmetric":5145},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1273,"cache_hit":0,"new_prefill":3190,"score_linear":15283.0,"score_lmetric":3190},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":138006,"ongoing_decode_tokens":138006,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1736,"cache_hit":0,"new_prefill":3190,"score_linear":138006.0,"score_lmetric":6380},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":111245,"ongoing_decode_tokens":0,"pending_prefill_tokens":11917,"num_requests":4,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":3190,"score_linear":111245.0,"score_lmetric":60428},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":246162,"ongoing_decode_tokens":0,"pending_prefill_tokens":164754,"num_requests":2,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":3190,"score_linear":246162.0,"score_lmetric":335888},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":68001,"ongoing_decode_tokens":13989,"pending_prefill_tokens":54012,"num_requests":2,"active_p_offloads":0,"cached_blocks":1327,"cache_hit":0,"new_prefill":3190,"score_linear":68001.0,"score_lmetric":114404},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":81806,"ongoing_decode_tokens":81806,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1341,"cache_hit":0,"new_prefill":3190,"score_linear":81806.0,"score_lmetric":6380},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":145668,"ongoing_decode_tokens":0,"pending_prefill_tokens":105220,"num_requests":2,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":3190,"score_linear":145668.0,"score_lmetric":216820}],"chosen_score_linear":15283.0,"chosen_score_lmetric":3190,"t_first_token":348815.356709098,"t_first_token_unix":1779779680.3706524,"t_done":348815.534853218,"t_done_unix":1779779680.5487964},{"request_id":"1382804:1:1382804:660","session_id":"1382804","input_length":627,"t_proxy_recv":348815.38226233,"t_decision_unix":1779779680.396204,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.125,"fallback_score":627,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":627,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":1955,"ongoing_decode_tokens":1955,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1400,"cache_hit":0,"new_prefill":627,"score_linear":1955.0,"score_lmetric":627},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":18473,"ongoing_decode_tokens":18473,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1273,"cache_hit":0,"new_prefill":627,"score_linear":18473.0,"score_lmetric":1254},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":138006,"ongoing_decode_tokens":138006,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1736,"cache_hit":0,"new_prefill":627,"score_linear":138006.0,"score_lmetric":1254},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":111245,"ongoing_decode_tokens":0,"pending_prefill_tokens":11917,"num_requests":4,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":627,"score_linear":111245.0,"score_lmetric":50176},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":246162,"ongoing_decode_tokens":0,"pending_prefill_tokens":164754,"num_requests":2,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":627,"score_linear":246162.0,"score_lmetric":330762},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":68001,"ongoing_decode_tokens":13989,"pending_prefill_tokens":54012,"num_requests":2,"active_p_offloads":0,"cached_blocks":1327,"cache_hit":0,"new_prefill":627,"score_linear":68001.0,"score_lmetric":109278},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":81806,"ongoing_decode_tokens":81806,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1341,"cache_hit":0,"new_prefill":627,"score_linear":81806.0,"score_lmetric":1254},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":145668,"ongoing_decode_tokens":0,"pending_prefill_tokens":105220,"num_requests":2,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":627,"score_linear":145668.0,"score_lmetric":211694}],"chosen_score_linear":1955.0,"chosen_score_lmetric":627,"t_first_token":348815.446684815,"t_first_token_unix":1779779680.4606278,"t_done":348815.6184267,"t_done_unix":1779779680.63237},{"request_id":"1366128:6:1375094:616","session_id":"1366128","input_length":58511,"t_proxy_recv":348808.736822348,"t_decision_unix":1779779673.750764,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":46592,"affinity_cache_ratio":0.7962947138144965,"affinity_num_requests":2,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"cache_hit":46592,"estimated_new_tokens":11919,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":35888,"ongoing_decode_tokens":35888,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1400,"cache_hit":0,"new_prefill":58511,"score_linear":35888.0,"score_lmetric":58511},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1266,"cache_hit":0,"new_prefill":58511,"score_linear":15283.0,"score_lmetric":58511},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":117536,"ongoing_decode_tokens":117536,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":46592,"new_prefill":11919,"score_linear":70944.0,"score_lmetric":23838},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":111245,"ongoing_decode_tokens":0,"pending_prefill_tokens":11917,"num_requests":4,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":58511,"score_linear":111245.0,"score_lmetric":281712},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":246162,"ongoing_decode_tokens":0,"pending_prefill_tokens":164754,"num_requests":2,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":58511,"score_linear":246162.0,"score_lmetric":446530},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1325,"cache_hit":0,"new_prefill":58511,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":125018,"ongoing_decode_tokens":0,"pending_prefill_tokens":46170,"num_requests":2,"active_p_offloads":0,"cached_blocks":1249,"cache_hit":0,"new_prefill":58511,"score_linear":125018.0,"score_lmetric":209362},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":98267,"ongoing_decode_tokens":0,"pending_prefill_tokens":85467,"num_requests":1,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":58511,"score_linear":98267.0,"score_lmetric":143978}],"chosen_score_linear":70944.0,"chosen_score_lmetric":23838,"t_first_token":348812.861300752,"t_first_token_unix":1779779677.8752444,"t_done":348816.606478377,"t_done_unix":1779779681.620422},{"request_id":"1382294:2:1383073:664","session_id":"1382294","input_length":5164,"t_proxy_recv":348816.249892681,"t_decision_unix":1779779681.263834,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":2560,"affinity_cache_ratio":0.4957397366382649,"affinity_num_requests":1,"avg_num_requests":2.0,"fallback_score":2604,"tie_break_used":false,"cache_hit":2560,"estimated_new_tokens":2604,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":5135,"ongoing_decode_tokens":0,"pending_prefill_tokens":5135,"num_requests":1,"active_p_offloads":0,"cached_blocks":1404,"cache_hit":0,"new_prefill":5164,"score_linear":5135.0,"score_lmetric":10299},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1279,"cache_hit":2560,"new_prefill":2604,"score_linear":12723.0,"score_lmetric":2604},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":138006,"ongoing_decode_tokens":138006,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1736,"cache_hit":0,"new_prefill":5164,"score_linear":138006.0,"score_lmetric":10328},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":111245,"ongoing_decode_tokens":0,"pending_prefill_tokens":11917,"num_requests":4,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":5164,"score_linear":111245.0,"score_lmetric":68324},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":246162,"ongoing_decode_tokens":0,"pending_prefill_tokens":164754,"num_requests":2,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":5164,"score_linear":246162.0,"score_lmetric":339836},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":68001,"ongoing_decode_tokens":13989,"pending_prefill_tokens":54012,"num_requests":2,"active_p_offloads":0,"cached_blocks":1327,"cache_hit":0,"new_prefill":5164,"score_linear":68001.0,"score_lmetric":118352},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":81806,"ongoing_decode_tokens":81806,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1341,"cache_hit":0,"new_prefill":5164,"score_linear":81806.0,"score_lmetric":10328},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":145668,"ongoing_decode_tokens":0,"pending_prefill_tokens":105220,"num_requests":2,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":5164,"score_linear":145668.0,"score_lmetric":220768}],"chosen_score_linear":12723.0,"chosen_score_lmetric":2604,"t_first_token":348816.426928375,"t_first_token_unix":1779779681.4408724,"t_done":348816.748861818,"t_done_unix":1779779681.762805},{"request_id":"1378543:2:1380447:644","session_id":"1378543","input_length":5314,"t_proxy_recv":348806.954501556,"t_decision_unix":1779779671.9684432,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":5120,"affinity_cache_ratio":0.963492660895747,"affinity_num_requests":3,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"cache_hit":5120,"estimated_new_tokens":194,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":59605,"ongoing_decode_tokens":59605,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1400,"cache_hit":0,"new_prefill":5314,"score_linear":59605.0,"score_lmetric":10628},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1266,"cache_hit":0,"new_prefill":5314,"score_linear":15283.0,"score_lmetric":5314},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":164300,"ongoing_decode_tokens":79495,"pending_prefill_tokens":13125,"num_requests":3,"active_p_offloads":0,"cached_blocks":1708,"cache_hit":0,"new_prefill":5314,"score_linear":164300.0,"score_lmetric":55317},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":105931,"ongoing_decode_tokens":0,"pending_prefill_tokens":11723,"num_requests":3,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":5120,"new_prefill":194,"score_linear":100811.0,"score_lmetric":35751},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":246162,"ongoing_decode_tokens":0,"pending_prefill_tokens":164754,"num_requests":2,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":5314,"score_linear":246162.0,"score_lmetric":340136},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1317,"cache_hit":0,"new_prefill":5314,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":125018,"ongoing_decode_tokens":0,"pending_prefill_tokens":46170,"num_requests":2,"active_p_offloads":0,"cached_blocks":1249,"cache_hit":0,"new_prefill":5314,"score_linear":125018.0,"score_lmetric":102968},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":98267,"ongoing_decode_tokens":0,"pending_prefill_tokens":85467,"num_requests":1,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":5314,"score_linear":98267.0,"score_lmetric":90781}],"chosen_score_linear":100811.0,"chosen_score_lmetric":35751,"t_first_token":348817.094070297,"t_first_token_unix":1779779682.1080134,"t_done":348817.103882516,"t_done_unix":1779779682.1178255},{"request_id":"1377061:3:1380174:641","session_id":"1377061","input_length":17368,"t_proxy_recv":348806.038814515,"t_decision_unix":1779779671.0527556,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":16384,"affinity_cache_ratio":0.943344081068632,"affinity_num_requests":1,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":16384,"estimated_new_tokens":984,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":59345,"ongoing_decode_tokens":59345,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1399,"cache_hit":0,"new_prefill":17368,"score_linear":59345.0,"score_lmetric":34736},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1266,"cache_hit":0,"new_prefill":17368,"score_linear":15283.0,"score_lmetric":17368},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":164300,"ongoing_decode_tokens":79495,"pending_prefill_tokens":13125,"num_requests":3,"active_p_offloads":0,"cached_blocks":1708,"cache_hit":0,"new_prefill":17368,"score_linear":164300.0,"score_lmetric":91479},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":57656,"ongoing_decode_tokens":0,"pending_prefill_tokens":1336,"num_requests":1,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":16384,"new_prefill":984,"score_linear":41272.0,"score_lmetric":2320},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":246162,"ongoing_decode_tokens":0,"pending_prefill_tokens":164754,"num_requests":2,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":17368,"score_linear":246162.0,"score_lmetric":364244},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1317,"cache_hit":0,"new_prefill":17368,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":125018,"ongoing_decode_tokens":0,"pending_prefill_tokens":46170,"num_requests":2,"active_p_offloads":0,"cached_blocks":1249,"cache_hit":0,"new_prefill":17368,"score_linear":125018.0,"score_lmetric":127076},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":98267,"ongoing_decode_tokens":0,"pending_prefill_tokens":85467,"num_requests":1,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":17368,"score_linear":98267.0,"score_lmetric":102835}],"chosen_score_linear":41272.0,"chosen_score_lmetric":2320,"t_first_token":348816.611550976,"t_first_token_unix":1779779681.625494,"t_done":348817.265751645,"t_done_unix":1779779682.2796948},{"request_id":"1383090:1:1383090:665","session_id":"1383090","input_length":9305,"t_proxy_recv":348816.291227539,"t_decision_unix":1779779681.305169,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.125,"fallback_score":14440,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":9305,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":5135,"ongoing_decode_tokens":0,"pending_prefill_tokens":5135,"num_requests":1,"active_p_offloads":0,"cached_blocks":1404,"cache_hit":0,"new_prefill":9305,"score_linear":5135.0,"score_lmetric":14440},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":20447,"ongoing_decode_tokens":15283,"pending_prefill_tokens":2604,"num_requests":2,"active_p_offloads":0,"cached_blocks":1279,"cache_hit":0,"new_prefill":9305,"score_linear":20447.0,"score_lmetric":23818},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":138006,"ongoing_decode_tokens":138006,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1736,"cache_hit":0,"new_prefill":9305,"score_linear":138006.0,"score_lmetric":18610},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":111245,"ongoing_decode_tokens":0,"pending_prefill_tokens":11917,"num_requests":4,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":9305,"score_linear":111245.0,"score_lmetric":84888},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":246162,"ongoing_decode_tokens":0,"pending_prefill_tokens":164754,"num_requests":2,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":9305,"score_linear":246162.0,"score_lmetric":348118},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":68001,"ongoing_decode_tokens":13989,"pending_prefill_tokens":54012,"num_requests":2,"active_p_offloads":0,"cached_blocks":1327,"cache_hit":0,"new_prefill":9305,"score_linear":68001.0,"score_lmetric":126634},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":81806,"ongoing_decode_tokens":81806,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1341,"cache_hit":0,"new_prefill":9305,"score_linear":81806.0,"score_lmetric":18610},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":145668,"ongoing_decode_tokens":0,"pending_prefill_tokens":105220,"num_requests":2,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":9305,"score_linear":145668.0,"score_lmetric":229050}],"chosen_score_linear":5135.0,"chosen_score_lmetric":14440,"t_first_token":348817.044314116,"t_first_token_unix":1779779682.0582573,"t_done":348817.319739921,"t_done_unix":1779779682.333683},{"request_id":"1383009:1:1383009:663","session_id":"1383009","input_length":5135,"t_proxy_recv":348816.025581651,"t_decision_unix":1779779681.039523,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.875,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":5135,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1404,"cache_hit":0,"new_prefill":5135,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1279,"cache_hit":0,"new_prefill":5135,"score_linear":15283.0,"score_lmetric":5135},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":138006,"ongoing_decode_tokens":138006,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1736,"cache_hit":0,"new_prefill":5135,"score_linear":138006.0,"score_lmetric":10270},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":111245,"ongoing_decode_tokens":0,"pending_prefill_tokens":11917,"num_requests":4,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":5135,"score_linear":111245.0,"score_lmetric":68208},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":246162,"ongoing_decode_tokens":0,"pending_prefill_tokens":164754,"num_requests":2,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":5135,"score_linear":246162.0,"score_lmetric":339778},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":68001,"ongoing_decode_tokens":13989,"pending_prefill_tokens":54012,"num_requests":2,"active_p_offloads":0,"cached_blocks":1327,"cache_hit":0,"new_prefill":5135,"score_linear":68001.0,"score_lmetric":118294},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":81806,"ongoing_decode_tokens":81806,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1341,"cache_hit":0,"new_prefill":5135,"score_linear":81806.0,"score_lmetric":10270},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":145668,"ongoing_decode_tokens":0,"pending_prefill_tokens":105220,"num_requests":2,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":5135,"score_linear":145668.0,"score_lmetric":220710}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348816.352560994,"t_first_token_unix":1779779681.3665042,"t_done":348817.340174566,"t_done_unix":1779779682.3541174},{"request_id":"1263749:2:1379302:636","session_id":"1263749","input_length":57656,"t_proxy_recv":348803.166274434,"t_decision_unix":1779779668.1802156,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":56320,"affinity_cache_ratio":0.9768280838074095,"affinity_num_requests":0,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":56320,"estimated_new_tokens":1336,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":58880,"ongoing_decode_tokens":58880,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1393,"cache_hit":0,"new_prefill":57656,"score_linear":58880.0,"score_lmetric":115312},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":15283,"ongoing_decode_tokens":0,"pending_prefill_tokens":15283,"num_requests":1,"active_p_offloads":0,"cached_blocks":1266,"cache_hit":0,"new_prefill":57656,"score_linear":15283.0,"score_lmetric":72939},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":164300,"ongoing_decode_tokens":0,"pending_prefill_tokens":92620,"num_requests":3,"active_p_offloads":0,"cached_blocks":1708,"cache_hit":0,"new_prefill":57656,"score_linear":164300.0,"score_lmetric":450828},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":56320,"new_prefill":1336,"score_linear":-56320.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":246162,"ongoing_decode_tokens":0,"pending_prefill_tokens":164754,"num_requests":2,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":57656,"score_linear":246162.0,"score_lmetric":444820},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1312,"cache_hit":0,"new_prefill":57656,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":125018,"ongoing_decode_tokens":0,"pending_prefill_tokens":46170,"num_requests":2,"active_p_offloads":0,"cached_blocks":1249,"cache_hit":0,"new_prefill":57656,"score_linear":125018.0,"score_lmetric":207652},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":98267,"ongoing_decode_tokens":0,"pending_prefill_tokens":85467,"num_requests":1,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":57656,"score_linear":98267.0,"score_lmetric":143123}],"chosen_score_linear":-56320.0,"chosen_score_lmetric":0,"t_first_token":348816.61051773,"t_first_token_unix":1779779681.6244607,"t_done":348817.663112313,"t_done_unix":1779779682.6770551},{"request_id":"1374465:3:1380413:642","session_id":"1374465","input_length":30907,"t_proxy_recv":348806.833698599,"t_decision_unix":1779779671.84764,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":21504,"affinity_cache_ratio":0.6957647134953246,"affinity_num_requests":2,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"cache_hit":21504,"estimated_new_tokens":9403,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":59605,"ongoing_decode_tokens":59605,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1400,"cache_hit":0,"new_prefill":30907,"score_linear":59605.0,"score_lmetric":61814},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1266,"cache_hit":0,"new_prefill":30907,"score_linear":15283.0,"score_lmetric":30907},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":164300,"ongoing_decode_tokens":79495,"pending_prefill_tokens":13125,"num_requests":3,"active_p_offloads":0,"cached_blocks":1708,"cache_hit":0,"new_prefill":30907,"score_linear":164300.0,"score_lmetric":132096},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":75024,"ongoing_decode_tokens":0,"pending_prefill_tokens":2320,"num_requests":2,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":21504,"new_prefill":9403,"score_linear":53520.0,"score_lmetric":23446},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":246162,"ongoing_decode_tokens":0,"pending_prefill_tokens":164754,"num_requests":2,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":30907,"score_linear":246162.0,"score_lmetric":391322},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1317,"cache_hit":0,"new_prefill":30907,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":125018,"ongoing_decode_tokens":0,"pending_prefill_tokens":46170,"num_requests":2,"active_p_offloads":0,"cached_blocks":1249,"cache_hit":0,"new_prefill":30907,"score_linear":125018.0,"score_lmetric":154154},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":98267,"ongoing_decode_tokens":0,"pending_prefill_tokens":85467,"num_requests":1,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":30907,"score_linear":98267.0,"score_lmetric":116374}],"chosen_score_linear":53520.0,"chosen_score_lmetric":23446,"t_first_token":348817.091242034,"t_first_token_unix":1779779682.1051853,"t_done":348817.950239473,"t_done_unix":1779779682.9641826},{"request_id":"1366128:7:1378071:626","session_id":"1366128","input_length":59007,"t_proxy_recv":348816.620878603,"t_decision_unix":1779779681.63482,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":58368,"affinity_cache_ratio":0.9891707763485688,"affinity_num_requests":1,"avg_num_requests":2.25,"fallback_score":null,"tie_break_used":false,"cache_hit":58368,"estimated_new_tokens":639,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":14440,"ongoing_decode_tokens":5135,"pending_prefill_tokens":9305,"num_requests":2,"active_p_offloads":0,"cached_blocks":1404,"cache_hit":0,"new_prefill":59007,"score_linear":14440.0,"score_lmetric":136624},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":20447,"ongoing_decode_tokens":20447,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1279,"cache_hit":0,"new_prefill":59007,"score_linear":20447.0,"score_lmetric":118014},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":79495,"ongoing_decode_tokens":79495,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1759,"cache_hit":58368,"new_prefill":639,"score_linear":21127.0,"score_lmetric":639},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":111245,"ongoing_decode_tokens":75024,"pending_prefill_tokens":9597,"num_requests":4,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":59007,"score_linear":111245.0,"score_lmetric":274416},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":246162,"ongoing_decode_tokens":0,"pending_prefill_tokens":164754,"num_requests":2,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":59007,"score_linear":246162.0,"score_lmetric":447522},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":78218,"ongoing_decode_tokens":13989,"pending_prefill_tokens":56549,"num_requests":3,"active_p_offloads":0,"cached_blocks":1327,"cache_hit":0,"new_prefill":59007,"score_linear":78218.0,"score_lmetric":346668},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":81806,"ongoing_decode_tokens":81806,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1341,"cache_hit":0,"new_prefill":59007,"score_linear":81806.0,"score_lmetric":118014},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":145668,"ongoing_decode_tokens":0,"pending_prefill_tokens":105220,"num_requests":2,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":59007,"score_linear":145668.0,"score_lmetric":328454}],"chosen_score_linear":21127.0,"chosen_score_lmetric":639,"t_first_token":348816.887923215,"t_first_token_unix":1779779681.9018662,"t_done":348818.147114859,"t_done_unix":1779779683.1610587},{"request_id":"1383625:1:1383625:668","session_id":"1383625","input_length":3186,"t_proxy_recv":348818.223880109,"t_decision_unix":1779779683.2378216,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.5,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":3186,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1432,"cache_hit":0,"new_prefill":3186,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1284,"cache_hit":0,"new_prefill":3186,"score_linear":15283.0,"score_lmetric":3186},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":138725,"ongoing_decode_tokens":79495,"pending_prefill_tokens":350,"num_requests":2,"active_p_offloads":0,"cached_blocks":1760,"cache_hit":0,"new_prefill":3186,"score_linear":138725.0,"score_lmetric":7072},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1887,"cache_hit":0,"new_prefill":3186,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":246162,"ongoing_decode_tokens":0,"pending_prefill_tokens":164754,"num_requests":2,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":3186,"score_linear":246162.0,"score_lmetric":335880},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":78218,"ongoing_decode_tokens":13989,"pending_prefill_tokens":56549,"num_requests":3,"active_p_offloads":0,"cached_blocks":1327,"cache_hit":0,"new_prefill":3186,"score_linear":78218.0,"score_lmetric":179205},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":81806,"ongoing_decode_tokens":81806,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1341,"cache_hit":0,"new_prefill":3186,"score_linear":81806.0,"score_lmetric":6372},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":145668,"ongoing_decode_tokens":0,"pending_prefill_tokens":105220,"num_requests":2,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":3186,"score_linear":145668.0,"score_lmetric":216812}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348818.419840665,"t_first_token_unix":1779779683.433784,"t_done":348818.551533721,"t_done_unix":1779779683.565477},{"request_id":"1373577:1:1373577:608","session_id":"1373577","input_length":79495,"t_proxy_recv":348783.29228177,"t_decision_unix":1779779648.306223,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":79495,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":304659,"ongoing_decode_tokens":240128,"pending_prefill_tokens":5651,"num_requests":4,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":2560,"new_prefill":76935,"score_linear":302099.0,"score_lmetric":330344},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":132609,"ongoing_decode_tokens":132609,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":79495,"score_linear":132609.0,"score_lmetric":158990},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1708,"cache_hit":0,"new_prefill":79495,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1750,"cache_hit":0,"new_prefill":79495,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":182626,"ongoing_decode_tokens":0,"pending_prefill_tokens":164194,"num_requests":1,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":79495,"score_linear":182626.0,"score_lmetric":243689},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1214,"cache_hit":0,"new_prefill":79495,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":60004,"ongoing_decode_tokens":60004,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1246,"cache_hit":0,"new_prefill":79495,"score_linear":60004.0,"score_lmetric":79495},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":106951,"ongoing_decode_tokens":106951,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1047,"cache_hit":0,"new_prefill":79495,"score_linear":106951.0,"score_lmetric":158990}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348805.250976112,"t_first_token_unix":1779779670.26492,"t_done":348818.588096782,"t_done_unix":1779779683.6020398},{"request_id":"1366128:8:1380438:643","session_id":"1366128","input_length":59230,"t_proxy_recv":348818.158969318,"t_decision_unix":1779779683.172911,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":58880,"affinity_cache_ratio":0.9940908323484721,"affinity_num_requests":1,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":58880,"estimated_new_tokens":350,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1432,"cache_hit":0,"new_prefill":59230,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1284,"cache_hit":0,"new_prefill":59230,"score_linear":15283.0,"score_lmetric":59230},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":79495,"ongoing_decode_tokens":79495,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1760,"cache_hit":58880,"new_prefill":350,"score_linear":20615.0,"score_lmetric":350},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1887,"cache_hit":0,"new_prefill":59230,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":246162,"ongoing_decode_tokens":0,"pending_prefill_tokens":164754,"num_requests":2,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":59230,"score_linear":246162.0,"score_lmetric":447968},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":78218,"ongoing_decode_tokens":13989,"pending_prefill_tokens":56549,"num_requests":3,"active_p_offloads":0,"cached_blocks":1327,"cache_hit":0,"new_prefill":59230,"score_linear":78218.0,"score_lmetric":347337},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":81806,"ongoing_decode_tokens":81806,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1341,"cache_hit":0,"new_prefill":59230,"score_linear":81806.0,"score_lmetric":118460},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":145668,"ongoing_decode_tokens":0,"pending_prefill_tokens":105220,"num_requests":2,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":59230,"score_linear":145668.0,"score_lmetric":328900}],"chosen_score_linear":20615.0,"chosen_score_lmetric":350,"t_first_token":348818.302517723,"t_first_token_unix":1779779683.3164608,"t_done":348820.068866209,"t_done_unix":1779779685.0828097},{"request_id":"1313181:5:1374956:615","session_id":"1313181","input_length":60858,"t_proxy_recv":348813.996488356,"t_decision_unix":1779779679.0104299,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":60416,"affinity_cache_ratio":0.9927371914949554,"affinity_num_requests":1,"avg_num_requests":2.25,"fallback_score":null,"tie_break_used":false,"cache_hit":60416,"estimated_new_tokens":442,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":35888,"ongoing_decode_tokens":35888,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1400,"cache_hit":24064,"new_prefill":36794,"score_linear":11824.0,"score_lmetric":36794},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":18317,"ongoing_decode_tokens":18317,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1268,"cache_hit":0,"new_prefill":60858,"score_linear":18317.0,"score_lmetric":121716},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":186762,"ongoing_decode_tokens":186762,"pending_prefill_tokens":0,"num_requests":4,"active_p_offloads":0,"cached_blocks":1733,"cache_hit":0,"new_prefill":60858,"score_linear":186762.0,"score_lmetric":243432},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":111245,"ongoing_decode_tokens":0,"pending_prefill_tokens":11917,"num_requests":4,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":60858,"score_linear":111245.0,"score_lmetric":291100},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":246162,"ongoing_decode_tokens":0,"pending_prefill_tokens":164754,"num_requests":2,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":60858,"score_linear":246162.0,"score_lmetric":451224},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":68001,"ongoing_decode_tokens":13989,"pending_prefill_tokens":54012,"num_requests":2,"active_p_offloads":0,"cached_blocks":1327,"cache_hit":0,"new_prefill":60858,"score_linear":68001.0,"score_lmetric":229740},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":20948,"ongoing_decode_tokens":20948,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1341,"cache_hit":60416,"new_prefill":442,"score_linear":-39468.0,"score_lmetric":442},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":145668,"ongoing_decode_tokens":0,"pending_prefill_tokens":105220,"num_requests":2,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":12800,"new_prefill":48058,"score_linear":132868.0,"score_lmetric":306556}],"chosen_score_linear":-39468.0,"chosen_score_lmetric":442,"t_first_token":348814.090187783,"t_first_token_unix":1779779679.1041312,"t_done":348821.056043579,"t_done_unix":1779779686.0699868},{"request_id":"1383625:2:1384279:673","session_id":"1383625","input_length":4150,"t_proxy_recv":348820.641640357,"t_decision_unix":1779779685.6555817,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":3072,"affinity_cache_ratio":0.7402409638554217,"affinity_num_requests":0,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"cache_hit":3072,"estimated_new_tokens":1078,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1438,"cache_hit":3072,"new_prefill":1078,"score_linear":-3072.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1284,"cache_hit":0,"new_prefill":4150,"score_linear":15283.0,"score_lmetric":4150},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":161742,"ongoing_decode_tokens":140521,"pending_prefill_tokens":229,"num_requests":3,"active_p_offloads":0,"cached_blocks":1916,"cache_hit":0,"new_prefill":4150,"score_linear":161742.0,"score_lmetric":13137},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":69465,"ongoing_decode_tokens":0,"pending_prefill_tokens":3417,"num_requests":1,"active_p_offloads":0,"cached_blocks":1887,"cache_hit":0,"new_prefill":4150,"score_linear":69465.0,"score_lmetric":7567},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":246162,"ongoing_decode_tokens":0,"pending_prefill_tokens":164754,"num_requests":2,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":4150,"score_linear":246162.0,"score_lmetric":337808},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":78218,"ongoing_decode_tokens":13989,"pending_prefill_tokens":56549,"num_requests":3,"active_p_offloads":0,"cached_blocks":1327,"cache_hit":0,"new_prefill":4150,"score_linear":78218.0,"score_lmetric":182097},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":81806,"ongoing_decode_tokens":81806,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1341,"cache_hit":0,"new_prefill":4150,"score_linear":81806.0,"score_lmetric":8300},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":145668,"ongoing_decode_tokens":0,"pending_prefill_tokens":105220,"num_requests":2,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":4150,"score_linear":145668.0,"score_lmetric":218740}],"chosen_score_linear":-3072.0,"chosen_score_lmetric":0,"t_first_token":348820.746171958,"t_first_token_unix":1779779685.7601151,"t_done":348821.175792935,"t_done_unix":1779779686.1897361},{"request_id":"1382667:2:1384299:674","session_id":"1382667","input_length":5989,"t_proxy_recv":348820.706188965,"t_decision_unix":1779779685.7201304,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":1536,"affinity_cache_ratio":0.2564701953581566,"affinity_num_requests":1,"avg_num_requests":1.875,"fallback_score":5531,"tie_break_used":false,"cache_hit":1536,"estimated_new_tokens":4453,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":4150,"ongoing_decode_tokens":0,"pending_prefill_tokens":1078,"num_requests":1,"active_p_offloads":0,"cached_blocks":1438,"cache_hit":1536,"new_prefill":4453,"score_linear":2614.0,"score_lmetric":5531},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1284,"cache_hit":0,"new_prefill":5989,"score_linear":15283.0,"score_lmetric":5989},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":161742,"ongoing_decode_tokens":140521,"pending_prefill_tokens":229,"num_requests":3,"active_p_offloads":0,"cached_blocks":1916,"cache_hit":0,"new_prefill":5989,"score_linear":161742.0,"score_lmetric":18654},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":69465,"ongoing_decode_tokens":0,"pending_prefill_tokens":3417,"num_requests":1,"active_p_offloads":0,"cached_blocks":1887,"cache_hit":0,"new_prefill":5989,"score_linear":69465.0,"score_lmetric":9406},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":246162,"ongoing_decode_tokens":0,"pending_prefill_tokens":164754,"num_requests":2,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":5989,"score_linear":246162.0,"score_lmetric":341486},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":78218,"ongoing_decode_tokens":13989,"pending_prefill_tokens":56549,"num_requests":3,"active_p_offloads":0,"cached_blocks":1327,"cache_hit":0,"new_prefill":5989,"score_linear":78218.0,"score_lmetric":187614},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":81806,"ongoing_decode_tokens":81806,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1341,"cache_hit":0,"new_prefill":5989,"score_linear":81806.0,"score_lmetric":11978},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":145668,"ongoing_decode_tokens":0,"pending_prefill_tokens":105220,"num_requests":2,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":5989,"score_linear":145668.0,"score_lmetric":222418}],"chosen_score_linear":2614.0,"chosen_score_lmetric":5531,"t_first_token":348821.038361767,"t_first_token_unix":1779779686.0523052,"t_done":348821.659152475,"t_done_unix":1779779686.6730957},{"request_id":"1384772:1:1384772:677","session_id":"1384772","input_length":4690,"t_proxy_recv":348822.169640005,"t_decision_unix":1779779687.1835814,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.0,"fallback_score":4690,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":4690,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25226,"ongoing_decode_tokens":25226,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1448,"cache_hit":0,"new_prefill":4690,"score_linear":25226.0,"score_lmetric":4690},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1284,"cache_hit":0,"new_prefill":4690,"score_linear":15283.0,"score_lmetric":4690},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":161742,"ongoing_decode_tokens":140521,"pending_prefill_tokens":229,"num_requests":3,"active_p_offloads":0,"cached_blocks":1916,"cache_hit":0,"new_prefill":4690,"score_linear":161742.0,"score_lmetric":14757},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":69465,"ongoing_decode_tokens":0,"pending_prefill_tokens":3417,"num_requests":1,"active_p_offloads":0,"cached_blocks":1887,"cache_hit":0,"new_prefill":4690,"score_linear":69465.0,"score_lmetric":8107},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":0,"pending_prefill_tokens":164955,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":4690,"score_linear":296027.0,"score_lmetric":508935},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":78218,"ongoing_decode_tokens":13989,"pending_prefill_tokens":56549,"num_requests":3,"active_p_offloads":0,"cached_blocks":1327,"cache_hit":0,"new_prefill":4690,"score_linear":78218.0,"score_lmetric":183717},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":127253,"ongoing_decode_tokens":20948,"pending_prefill_tokens":44865,"num_requests":2,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":4690,"score_linear":127253.0,"score_lmetric":99110},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":145668,"ongoing_decode_tokens":0,"pending_prefill_tokens":105220,"num_requests":2,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":4690,"score_linear":145668.0,"score_lmetric":219820}],"chosen_score_linear":15283.0,"chosen_score_lmetric":4690,"t_first_token":348822.469950508,"t_first_token_unix":1779779687.4838939,"t_done":348822.658024227,"t_done_unix":1779779687.6719685},{"request_id":"1366128:9:1381799:652","session_id":"1366128","input_length":59374,"t_proxy_recv":348820.078944874,"t_decision_unix":1779779685.0928862,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":58880,"affinity_cache_ratio":0.9916798598713241,"affinity_num_requests":1,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":58880,"estimated_new_tokens":494,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1438,"cache_hit":0,"new_prefill":59374,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1284,"cache_hit":0,"new_prefill":59374,"score_linear":15283.0,"score_lmetric":59374},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":81147,"ongoing_decode_tokens":81147,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1916,"cache_hit":58880,"new_prefill":494,"score_linear":22267.0,"score_lmetric":494},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":69465,"ongoing_decode_tokens":0,"pending_prefill_tokens":3417,"num_requests":1,"active_p_offloads":0,"cached_blocks":1887,"cache_hit":0,"new_prefill":59374,"score_linear":69465.0,"score_lmetric":62791},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":246162,"ongoing_decode_tokens":0,"pending_prefill_tokens":164754,"num_requests":2,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":59374,"score_linear":246162.0,"score_lmetric":448256},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":78218,"ongoing_decode_tokens":13989,"pending_prefill_tokens":56549,"num_requests":3,"active_p_offloads":0,"cached_blocks":1327,"cache_hit":0,"new_prefill":59374,"score_linear":78218.0,"score_lmetric":347769},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":81806,"ongoing_decode_tokens":81806,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1341,"cache_hit":0,"new_prefill":59374,"score_linear":81806.0,"score_lmetric":118748},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":145668,"ongoing_decode_tokens":0,"pending_prefill_tokens":105220,"num_requests":2,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":59374,"score_linear":145668.0,"score_lmetric":329188}],"chosen_score_linear":22267.0,"chosen_score_lmetric":494,"t_first_token":348820.199311195,"t_first_token_unix":1779779685.2132552,"t_done":348823.138487217,"t_done_unix":1779779688.1524305},{"request_id":"1383009:2:1384930:679","session_id":"1383009","input_length":11113,"t_proxy_recv":348822.677458409,"t_decision_unix":1779779687.6914,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":5120,"affinity_cache_ratio":0.46072167731485647,"affinity_num_requests":1,"avg_num_requests":2.0,"fallback_score":5993,"tie_break_used":false,"cache_hit":5120,"estimated_new_tokens":5993,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":25226,"ongoing_decode_tokens":25226,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1448,"cache_hit":5120,"new_prefill":5993,"score_linear":20106.0,"score_lmetric":5993},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1293,"cache_hit":0,"new_prefill":11113,"score_linear":15283.0,"score_lmetric":11113},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":161742,"ongoing_decode_tokens":161742,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1916,"cache_hit":0,"new_prefill":11113,"score_linear":161742.0,"score_lmetric":33339},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":69465,"ongoing_decode_tokens":0,"pending_prefill_tokens":3417,"num_requests":1,"active_p_offloads":0,"cached_blocks":1887,"cache_hit":0,"new_prefill":11113,"score_linear":69465.0,"score_lmetric":14530},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":0,"pending_prefill_tokens":164955,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":11113,"score_linear":296027.0,"score_lmetric":528204},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":78218,"ongoing_decode_tokens":13989,"pending_prefill_tokens":56549,"num_requests":3,"active_p_offloads":0,"cached_blocks":1327,"cache_hit":0,"new_prefill":11113,"score_linear":78218.0,"score_lmetric":202986},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":127253,"ongoing_decode_tokens":20948,"pending_prefill_tokens":44865,"num_requests":2,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":11113,"score_linear":127253.0,"score_lmetric":111956},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":145668,"ongoing_decode_tokens":0,"pending_prefill_tokens":105220,"num_requests":2,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":11113,"score_linear":145668.0,"score_lmetric":232666}],"chosen_score_linear":20106.0,"chosen_score_lmetric":5993,"t_first_token":348823.247526875,"t_first_token_unix":1779779688.2614698,"t_done":348823.476428028,"t_done_unix":1779779688.490371},{"request_id":"1355484:10:1384484:675","session_id":"1355484","input_length":25226,"t_proxy_recv":348821.309870374,"t_decision_unix":1779779686.3238115,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":23552,"affinity_cache_ratio":0.9336398953460715,"affinity_num_requests":1,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"cache_hit":23552,"estimated_new_tokens":1674,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":5989,"ongoing_decode_tokens":5989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1440,"cache_hit":23552,"new_prefill":1674,"score_linear":-17563.0,"score_lmetric":1674},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1284,"cache_hit":0,"new_prefill":25226,"score_linear":15283.0,"score_lmetric":25226},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":161742,"ongoing_decode_tokens":140521,"pending_prefill_tokens":229,"num_requests":3,"active_p_offloads":0,"cached_blocks":1916,"cache_hit":0,"new_prefill":25226,"score_linear":161742.0,"score_lmetric":76365},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":69465,"ongoing_decode_tokens":0,"pending_prefill_tokens":3417,"num_requests":1,"active_p_offloads":0,"cached_blocks":1887,"cache_hit":0,"new_prefill":25226,"score_linear":69465.0,"score_lmetric":28643},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":246162,"ongoing_decode_tokens":0,"pending_prefill_tokens":164754,"num_requests":2,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":25226,"score_linear":246162.0,"score_lmetric":379960},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":78218,"ongoing_decode_tokens":13989,"pending_prefill_tokens":56549,"num_requests":3,"active_p_offloads":0,"cached_blocks":1327,"cache_hit":0,"new_prefill":25226,"score_linear":78218.0,"score_lmetric":245325},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":127253,"ongoing_decode_tokens":20948,"pending_prefill_tokens":44865,"num_requests":2,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":12800,"new_prefill":12426,"score_linear":114453.0,"score_lmetric":114582},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":145668,"ongoing_decode_tokens":0,"pending_prefill_tokens":105220,"num_requests":2,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":12800,"new_prefill":12426,"score_linear":132868.0,"score_lmetric":235292}],"chosen_score_linear":-17563.0,"chosen_score_lmetric":1674,"t_first_token":348821.63633925,"t_first_token_unix":1779779686.6502824,"t_done":348823.70340716,"t_done_unix":1779779688.717352},{"request_id":"1366128:10:1382843:661","session_id":"1366128","input_length":59506,"t_proxy_recv":348823.148790401,"t_decision_unix":1779779688.1627321,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":59392,"affinity_cache_ratio":0.9980842268006588,"affinity_num_requests":2,"avg_num_requests":2.125,"fallback_score":null,"tie_break_used":false,"cache_hit":59392,"estimated_new_tokens":114,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":36339,"ongoing_decode_tokens":25226,"pending_prefill_tokens":5993,"num_requests":2,"active_p_offloads":0,"cached_blocks":1448,"cache_hit":0,"new_prefill":59506,"score_linear":36339.0,"score_lmetric":130998},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":34317,"ongoing_decode_tokens":15283,"pending_prefill_tokens":19034,"num_requests":2,"active_p_offloads":0,"cached_blocks":1293,"cache_hit":0,"new_prefill":59506,"score_linear":34317.0,"score_lmetric":157080},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":102368,"ongoing_decode_tokens":102368,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1917,"cache_hit":59392,"new_prefill":114,"score_linear":42976.0,"score_lmetric":228},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":69465,"ongoing_decode_tokens":0,"pending_prefill_tokens":3417,"num_requests":1,"active_p_offloads":0,"cached_blocks":1887,"cache_hit":0,"new_prefill":59506,"score_linear":69465.0,"score_lmetric":62923},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":0,"pending_prefill_tokens":164955,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":59506,"score_linear":296027.0,"score_lmetric":673383},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":78218,"ongoing_decode_tokens":13989,"pending_prefill_tokens":56549,"num_requests":3,"active_p_offloads":0,"cached_blocks":1327,"cache_hit":0,"new_prefill":59506,"score_linear":78218.0,"score_lmetric":348165},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":127253,"ongoing_decode_tokens":20948,"pending_prefill_tokens":44865,"num_requests":2,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":59506,"score_linear":127253.0,"score_lmetric":208742},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":145668,"ongoing_decode_tokens":0,"pending_prefill_tokens":105220,"num_requests":2,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":59506,"score_linear":145668.0,"score_lmetric":329452}],"chosen_score_linear":42976.0,"chosen_score_lmetric":228,"t_first_token":348823.302120641,"t_first_token_unix":1779779688.3160634,"t_done":348824.295681826,"t_done_unix":1779779689.3096256},{"request_id":"1381387:1:1381387:648","session_id":"1381387","input_length":13989,"t_proxy_recv":348810.316058862,"t_decision_unix":1779779675.33,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.75,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":13989,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":35888,"ongoing_decode_tokens":35888,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1400,"cache_hit":0,"new_prefill":13989,"score_linear":35888.0,"score_lmetric":13989},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1266,"cache_hit":0,"new_prefill":13989,"score_linear":15283.0,"score_lmetric":13989},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":176047,"ongoing_decode_tokens":117536,"pending_prefill_tokens":11919,"num_requests":3,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":13989,"score_linear":176047.0,"score_lmetric":77724},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":111245,"ongoing_decode_tokens":0,"pending_prefill_tokens":11917,"num_requests":4,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":13989,"score_linear":111245.0,"score_lmetric":103624},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":246162,"ongoing_decode_tokens":0,"pending_prefill_tokens":164754,"num_requests":2,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":13989,"score_linear":246162.0,"score_lmetric":357486},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1325,"cache_hit":0,"new_prefill":13989,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":125018,"ongoing_decode_tokens":125018,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1249,"cache_hit":0,"new_prefill":13989,"score_linear":125018.0,"score_lmetric":27978},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":98267,"ongoing_decode_tokens":0,"pending_prefill_tokens":85467,"num_requests":1,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":13989,"score_linear":98267.0,"score_lmetric":99456}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348811.514646535,"t_first_token_unix":1779779676.5285897,"t_done":348824.305444871,"t_done_unix":1779779689.3193882},{"request_id":"1378979:1:1378979:633","session_id":"1378979","input_length":15283,"t_proxy_recv":348801.951837752,"t_decision_unix":1779779666.9657793,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.5,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":15283,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":58880,"ongoing_decode_tokens":58880,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1393,"cache_hit":0,"new_prefill":15283,"score_linear":58880.0,"score_lmetric":30566},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1266,"cache_hit":0,"new_prefill":15283,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":164300,"ongoing_decode_tokens":0,"pending_prefill_tokens":92620,"num_requests":3,"active_p_offloads":0,"cached_blocks":1708,"cache_hit":0,"new_prefill":15283,"score_linear":164300.0,"score_lmetric":323709},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":5458,"ongoing_decode_tokens":5458,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1853,"cache_hit":14848,"new_prefill":435,"score_linear":-9390.0,"score_lmetric":435},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":246162,"ongoing_decode_tokens":0,"pending_prefill_tokens":164754,"num_requests":2,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":15283,"score_linear":246162.0,"score_lmetric":360074},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":22132,"ongoing_decode_tokens":22132,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1299,"cache_hit":0,"new_prefill":15283,"score_linear":22132.0,"score_lmetric":15283},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":125018,"ongoing_decode_tokens":0,"pending_prefill_tokens":46170,"num_requests":2,"active_p_offloads":0,"cached_blocks":1249,"cache_hit":0,"new_prefill":15283,"score_linear":125018.0,"score_lmetric":122906},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":98267,"ongoing_decode_tokens":0,"pending_prefill_tokens":85467,"num_requests":1,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":15283,"score_linear":98267.0,"score_lmetric":100750}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348803.314428359,"t_first_token_unix":1779779668.328372,"t_done":348824.883668642,"t_done_unix":1779779689.8976119},{"request_id":"1384947:1:1384947:680","session_id":"1384947","input_length":19034,"t_proxy_recv":348822.736092047,"t_decision_unix":1779779687.7500336,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.125,"fallback_score":19034,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":19034,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":36339,"ongoing_decode_tokens":25226,"pending_prefill_tokens":5993,"num_requests":2,"active_p_offloads":0,"cached_blocks":1448,"cache_hit":0,"new_prefill":19034,"score_linear":36339.0,"score_lmetric":50054},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1293,"cache_hit":0,"new_prefill":19034,"score_linear":15283.0,"score_lmetric":19034},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":161742,"ongoing_decode_tokens":161742,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1916,"cache_hit":0,"new_prefill":19034,"score_linear":161742.0,"score_lmetric":57102},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":69465,"ongoing_decode_tokens":0,"pending_prefill_tokens":3417,"num_requests":1,"active_p_offloads":0,"cached_blocks":1887,"cache_hit":0,"new_prefill":19034,"score_linear":69465.0,"score_lmetric":22451},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":0,"pending_prefill_tokens":164955,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":19034,"score_linear":296027.0,"score_lmetric":551967},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":78218,"ongoing_decode_tokens":13989,"pending_prefill_tokens":56549,"num_requests":3,"active_p_offloads":0,"cached_blocks":1327,"cache_hit":0,"new_prefill":19034,"score_linear":78218.0,"score_lmetric":226749},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":127253,"ongoing_decode_tokens":20948,"pending_prefill_tokens":44865,"num_requests":2,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":19034,"score_linear":127253.0,"score_lmetric":127798},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":145668,"ongoing_decode_tokens":0,"pending_prefill_tokens":105220,"num_requests":2,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":19034,"score_linear":145668.0,"score_lmetric":248508}],"chosen_score_linear":15283.0,"chosen_score_lmetric":19034,"t_first_token":348824.661796315,"t_first_token_unix":1779779689.67574,"t_done":348824.979278641,"t_done_unix":1779779689.9932215},{"request_id":"1383090:2:1385262:682","session_id":"1383090","input_length":16734,"t_proxy_recv":348823.925613057,"t_decision_unix":1779779688.9395442,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":9216,"affinity_cache_ratio":0.5507350304768734,"affinity_num_requests":0,"avg_num_requests":2.0,"fallback_score":null,"tie_break_used":false,"cache_hit":9216,"estimated_new_tokens":7518,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1462,"cache_hit":9216,"new_prefill":7518,"score_linear":-9216.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":34317,"ongoing_decode_tokens":15283,"pending_prefill_tokens":19034,"num_requests":2,"active_p_offloads":0,"cached_blocks":1293,"cache_hit":0,"new_prefill":16734,"score_linear":34317.0,"score_lmetric":71536},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":161874,"ongoing_decode_tokens":161874,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1917,"cache_hit":0,"new_prefill":16734,"score_linear":161874.0,"score_lmetric":50202},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":69465,"ongoing_decode_tokens":0,"pending_prefill_tokens":3417,"num_requests":1,"active_p_offloads":0,"cached_blocks":1887,"cache_hit":0,"new_prefill":16734,"score_linear":69465.0,"score_lmetric":20151},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":0,"pending_prefill_tokens":164955,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":16734,"score_linear":296027.0,"score_lmetric":545067},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":78218,"ongoing_decode_tokens":78218,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1327,"cache_hit":0,"new_prefill":16734,"score_linear":78218.0,"score_lmetric":50202},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":127253,"ongoing_decode_tokens":20948,"pending_prefill_tokens":44865,"num_requests":2,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":16734,"score_linear":127253.0,"score_lmetric":123198},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":145668,"ongoing_decode_tokens":0,"pending_prefill_tokens":105220,"num_requests":2,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":16734,"score_linear":145668.0,"score_lmetric":243908}],"chosen_score_linear":-9216.0,"chosen_score_lmetric":0,"t_first_token":348824.823883654,"t_first_token_unix":1779779689.837827,"t_done":348825.080372943,"t_done_unix":1779779690.094316},{"request_id":"1379569:4:1383116:666","session_id":"1379569","input_length":10217,"t_proxy_recv":348816.381886435,"t_decision_unix":1779779681.395828,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":7680,"affinity_cache_ratio":0.7516883625330332,"affinity_num_requests":2,"avg_num_requests":2.25,"fallback_score":null,"tie_break_used":false,"cache_hit":7680,"estimated_new_tokens":2537,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":14440,"ongoing_decode_tokens":5135,"pending_prefill_tokens":9305,"num_requests":2,"active_p_offloads":0,"cached_blocks":1404,"cache_hit":0,"new_prefill":10217,"score_linear":14440.0,"score_lmetric":39044},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":20447,"ongoing_decode_tokens":15283,"pending_prefill_tokens":2604,"num_requests":2,"active_p_offloads":0,"cached_blocks":1279,"cache_hit":0,"new_prefill":10217,"score_linear":20447.0,"score_lmetric":25642},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":138006,"ongoing_decode_tokens":138006,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1736,"cache_hit":0,"new_prefill":10217,"score_linear":138006.0,"score_lmetric":20434},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":111245,"ongoing_decode_tokens":0,"pending_prefill_tokens":11917,"num_requests":4,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":10217,"score_linear":111245.0,"score_lmetric":88536},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":246162,"ongoing_decode_tokens":0,"pending_prefill_tokens":164754,"num_requests":2,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":10217,"score_linear":246162.0,"score_lmetric":349942},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":68001,"ongoing_decode_tokens":13989,"pending_prefill_tokens":54012,"num_requests":2,"active_p_offloads":0,"cached_blocks":1327,"cache_hit":7680,"new_prefill":2537,"score_linear":60321.0,"score_lmetric":113098},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":81806,"ongoing_decode_tokens":81806,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1341,"cache_hit":0,"new_prefill":10217,"score_linear":81806.0,"score_lmetric":20434},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":145668,"ongoing_decode_tokens":0,"pending_prefill_tokens":105220,"num_requests":2,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":10217,"score_linear":145668.0,"score_lmetric":230874}],"chosen_score_linear":60321.0,"chosen_score_lmetric":113098,"t_first_token":348823.723194108,"t_first_token_unix":1779779688.737137,"t_done":348826.731874238,"t_done_unix":1779779691.7458172},{"request_id":"1382086:1:1382086:654","session_id":"1382086","input_length":54012,"t_proxy_recv":348812.916724083,"t_decision_unix":1779779677.930666,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.25,"fallback_score":54012,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":54012,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":35888,"ongoing_decode_tokens":35888,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1400,"cache_hit":0,"new_prefill":54012,"score_linear":35888.0,"score_lmetric":54012},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1268,"cache_hit":0,"new_prefill":54012,"score_linear":15283.0,"score_lmetric":54012},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":224803,"ongoing_decode_tokens":176047,"pending_prefill_tokens":3700,"num_requests":5,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":54012,"score_linear":224803.0,"score_lmetric":288560},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":111245,"ongoing_decode_tokens":0,"pending_prefill_tokens":11917,"num_requests":4,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":54012,"score_linear":111245.0,"score_lmetric":263716},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":246162,"ongoing_decode_tokens":0,"pending_prefill_tokens":164754,"num_requests":2,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":54012,"score_linear":246162.0,"score_lmetric":437532},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":13989,"ongoing_decode_tokens":13989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1327,"cache_hit":0,"new_prefill":54012,"score_linear":13989.0,"score_lmetric":54012},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":126629,"ongoing_decode_tokens":126629,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1250,"cache_hit":0,"new_prefill":54012,"score_linear":126629.0,"score_lmetric":108024},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":145668,"ongoing_decode_tokens":0,"pending_prefill_tokens":105220,"num_requests":2,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":54012,"score_linear":145668.0,"score_lmetric":318464}],"chosen_score_linear":13989.0,"chosen_score_lmetric":54012,"t_first_token":348823.721536387,"t_first_token_unix":1779779688.7354796,"t_done":348827.687768948,"t_done_unix":1779779692.7017124},{"request_id":"1381387:2:1383735:669","session_id":"1381387","input_length":28383,"t_proxy_recv":348824.334247439,"t_decision_unix":1779779689.3481889,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":13824,"affinity_cache_ratio":0.4870521086565902,"affinity_num_requests":2,"avg_num_requests":2.0,"fallback_score":29118,"tie_break_used":false,"cache_hit":13824,"estimated_new_tokens":14559,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":16734,"ongoing_decode_tokens":0,"pending_prefill_tokens":7518,"num_requests":1,"active_p_offloads":0,"cached_blocks":1462,"cache_hit":0,"new_prefill":28383,"score_linear":16734.0,"score_lmetric":35901},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":34317,"ongoing_decode_tokens":15283,"pending_prefill_tokens":19034,"num_requests":2,"active_p_offloads":0,"cached_blocks":1293,"cache_hit":0,"new_prefill":28383,"score_linear":34317.0,"score_lmetric":94834},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":173622,"ongoing_decode_tokens":102368,"pending_prefill_tokens":11862,"num_requests":3,"active_p_offloads":0,"cached_blocks":1917,"cache_hit":0,"new_prefill":28383,"score_linear":173622.0,"score_lmetric":120735},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":69465,"ongoing_decode_tokens":0,"pending_prefill_tokens":3417,"num_requests":1,"active_p_offloads":0,"cached_blocks":1887,"cache_hit":0,"new_prefill":28383,"score_linear":69465.0,"score_lmetric":31800},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":0,"pending_prefill_tokens":164955,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":28383,"score_linear":296027.0,"score_lmetric":580014},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":64229,"ongoing_decode_tokens":64229,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1353,"cache_hit":13824,"new_prefill":14559,"score_linear":50405.0,"score_lmetric":29118},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":127253,"ongoing_decode_tokens":20948,"pending_prefill_tokens":44865,"num_requests":2,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":28383,"score_linear":127253.0,"score_lmetric":146496},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":145668,"ongoing_decode_tokens":0,"pending_prefill_tokens":105220,"num_requests":2,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":28383,"score_linear":145668.0,"score_lmetric":267206}],"chosen_score_linear":50405.0,"chosen_score_lmetric":29118,"t_first_token":348826.710589061,"t_first_token_unix":1779779691.724533,"t_done":348828.884023168,"t_done_unix":1779779693.897967},{"request_id":"1381728:2:1387040:690","session_id":"1381728","input_length":12399,"t_proxy_recv":348829.959096423,"t_decision_unix":1779779694.9730375,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":1024,"affinity_cache_ratio":0.08258730542785708,"affinity_num_requests":0,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":false,"cache_hit":1024,"estimated_new_tokens":11375,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1476,"cache_hit":0,"new_prefill":12399,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1365,"cache_hit":1024,"new_prefill":11375,"score_linear":-1024.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":173622,"ongoing_decode_tokens":173622,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1917,"cache_hit":0,"new_prefill":12399,"score_linear":173622.0,"score_lmetric":37197},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":69465,"ongoing_decode_tokens":0,"pending_prefill_tokens":3417,"num_requests":1,"active_p_offloads":0,"cached_blocks":1887,"cache_hit":0,"new_prefill":12399,"score_linear":69465.0,"score_lmetric":15816},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":0,"pending_prefill_tokens":164955,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":12399,"score_linear":296027.0,"score_lmetric":532062},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1491,"cache_hit":0,"new_prefill":12399,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":127253,"ongoing_decode_tokens":20948,"pending_prefill_tokens":44865,"num_requests":2,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":12399,"score_linear":127253.0,"score_lmetric":114528},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":145668,"ongoing_decode_tokens":98267,"pending_prefill_tokens":19753,"num_requests":2,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":12399,"score_linear":145668.0,"score_lmetric":64304}],"chosen_score_linear":-1024.0,"chosen_score_lmetric":0,"t_first_token":348830.913577294,"t_first_token_unix":1779779695.9275208,"t_done":348830.99784892,"t_done_unix":1779779696.0117922},{"request_id":"1340290:3:1384149:672","session_id":"1340290","input_length":21221,"t_proxy_recv":348820.164640891,"t_decision_unix":1779779685.1785822,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":20992,"affinity_cache_ratio":0.9892088026011969,"affinity_num_requests":2,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"cache_hit":20992,"estimated_new_tokens":229,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1438,"cache_hit":0,"new_prefill":21221,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1284,"cache_hit":0,"new_prefill":21221,"score_linear":15283.0,"score_lmetric":21221},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":140521,"ongoing_decode_tokens":81147,"pending_prefill_tokens":494,"num_requests":2,"active_p_offloads":0,"cached_blocks":1916,"cache_hit":20992,"new_prefill":229,"score_linear":119529.0,"score_lmetric":1446},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":69465,"ongoing_decode_tokens":0,"pending_prefill_tokens":3417,"num_requests":1,"active_p_offloads":0,"cached_blocks":1887,"cache_hit":0,"new_prefill":21221,"score_linear":69465.0,"score_lmetric":24638},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":246162,"ongoing_decode_tokens":0,"pending_prefill_tokens":164754,"num_requests":2,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":21221,"score_linear":246162.0,"score_lmetric":371950},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":78218,"ongoing_decode_tokens":13989,"pending_prefill_tokens":56549,"num_requests":3,"active_p_offloads":0,"cached_blocks":1327,"cache_hit":0,"new_prefill":21221,"score_linear":78218.0,"score_lmetric":233310},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":81806,"ongoing_decode_tokens":81806,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1341,"cache_hit":1024,"new_prefill":20197,"score_linear":80782.0,"score_lmetric":40394},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":145668,"ongoing_decode_tokens":0,"pending_prefill_tokens":105220,"num_requests":2,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":21221,"score_linear":145668.0,"score_lmetric":252882}],"chosen_score_linear":119529.0,"chosen_score_lmetric":1446,"t_first_token":348822.512977883,"t_first_token_unix":1779779687.526921,"t_done":348831.016082517,"t_done_unix":1779779696.0300255},{"request_id":"1384772:2:1387335:693","session_id":"1384772","input_length":11530,"t_proxy_recv":348831.057866608,"t_decision_unix":1779779696.0718083,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":4608,"affinity_cache_ratio":0.39965307892454466,"affinity_num_requests":0,"avg_num_requests":1.5,"fallback_score":0,"tie_break_used":false,"cache_hit":4608,"estimated_new_tokens":6922,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1476,"cache_hit":0,"new_prefill":11530,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1387,"cache_hit":4608,"new_prefill":6922,"score_linear":-4608.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":178547,"ongoing_decode_tokens":152401,"pending_prefill_tokens":5154,"num_requests":3,"active_p_offloads":0,"cached_blocks":1917,"cache_hit":0,"new_prefill":11530,"score_linear":178547.0,"score_lmetric":50052},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":69465,"ongoing_decode_tokens":0,"pending_prefill_tokens":3417,"num_requests":1,"active_p_offloads":0,"cached_blocks":1887,"cache_hit":0,"new_prefill":11530,"score_linear":69465.0,"score_lmetric":14947},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":0,"pending_prefill_tokens":164955,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":11530,"score_linear":296027.0,"score_lmetric":529455},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":50793,"ongoing_decode_tokens":0,"pending_prefill_tokens":22633,"num_requests":1,"active_p_offloads":0,"cached_blocks":1491,"cache_hit":0,"new_prefill":11530,"score_linear":50793.0,"score_lmetric":34163},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":127253,"ongoing_decode_tokens":20948,"pending_prefill_tokens":44865,"num_requests":2,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":11530,"score_linear":127253.0,"score_lmetric":112790},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":145668,"ongoing_decode_tokens":98267,"pending_prefill_tokens":19753,"num_requests":2,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":11530,"score_linear":145668.0,"score_lmetric":62566}],"chosen_score_linear":-4608.0,"chosen_score_lmetric":0,"t_first_token":348831.698788301,"t_first_token_unix":1779779696.7127316,"t_done":348831.758510766,"t_done_unix":1779779696.7724543},{"request_id":"1340290:4:1385568:684","session_id":"1340290","input_length":26146,"t_proxy_recv":348831.023683733,"t_decision_unix":1779779696.0376253,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":20992,"affinity_cache_ratio":0.8028761569647365,"affinity_num_requests":2,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":20992,"estimated_new_tokens":5154,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1476,"cache_hit":0,"new_prefill":26146,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1387,"cache_hit":0,"new_prefill":26146,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":152401,"ongoing_decode_tokens":152401,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1917,"cache_hit":20992,"new_prefill":5154,"score_linear":131409.0,"score_lmetric":10308},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":69465,"ongoing_decode_tokens":0,"pending_prefill_tokens":3417,"num_requests":1,"active_p_offloads":0,"cached_blocks":1887,"cache_hit":0,"new_prefill":26146,"score_linear":69465.0,"score_lmetric":29563},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":0,"pending_prefill_tokens":164955,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":26146,"score_linear":296027.0,"score_lmetric":573303},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":50793,"ongoing_decode_tokens":0,"pending_prefill_tokens":22633,"num_requests":1,"active_p_offloads":0,"cached_blocks":1491,"cache_hit":0,"new_prefill":26146,"score_linear":50793.0,"score_lmetric":48779},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":127253,"ongoing_decode_tokens":20948,"pending_prefill_tokens":44865,"num_requests":2,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":1024,"new_prefill":25122,"score_linear":126229.0,"score_lmetric":139974},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":145668,"ongoing_decode_tokens":98267,"pending_prefill_tokens":19753,"num_requests":2,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":26146,"score_linear":145668.0,"score_lmetric":91798}],"chosen_score_linear":131409.0,"chosen_score_lmetric":10308,"t_first_token":348831.942179179,"t_first_token_unix":1779779696.9561224,"t_done":348833.170528819,"t_done_unix":1779779698.184472},{"request_id":"1373577:2:1380740:647","session_id":"1373577","input_length":81147,"t_proxy_recv":348818.602467625,"t_decision_unix":1779779683.6164093,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":79872,"affinity_cache_ratio":0.9842877740397057,"affinity_num_requests":1,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":79872,"estimated_new_tokens":1275,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1438,"cache_hit":2560,"new_prefill":78587,"score_linear":-2560.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1284,"cache_hit":0,"new_prefill":81147,"score_linear":15283.0,"score_lmetric":81147},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":59230,"ongoing_decode_tokens":59230,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1916,"cache_hit":79872,"new_prefill":1275,"score_linear":-20642.0,"score_lmetric":1275},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1887,"cache_hit":0,"new_prefill":81147,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":246162,"ongoing_decode_tokens":0,"pending_prefill_tokens":164754,"num_requests":2,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":81147,"score_linear":246162.0,"score_lmetric":491802},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":78218,"ongoing_decode_tokens":13989,"pending_prefill_tokens":56549,"num_requests":3,"active_p_offloads":0,"cached_blocks":1327,"cache_hit":0,"new_prefill":81147,"score_linear":78218.0,"score_lmetric":413088},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":81806,"ongoing_decode_tokens":81806,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1341,"cache_hit":0,"new_prefill":81147,"score_linear":81806.0,"score_lmetric":162294},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":145668,"ongoing_decode_tokens":0,"pending_prefill_tokens":105220,"num_requests":2,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":81147,"score_linear":145668.0,"score_lmetric":372734}],"chosen_score_linear":-20642.0,"chosen_score_lmetric":1275,"t_first_token":348819.453268082,"t_first_token_unix":1779779684.467211,"t_done":348833.713641495,"t_done_unix":1779779698.7275848},{"request_id":"1366128:11:1384044:671","session_id":"1366128","input_length":71254,"t_proxy_recv":348824.326058618,"t_decision_unix":1779779689.3400002,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":59392,"affinity_cache_ratio":0.8335251354309934,"affinity_num_requests":2,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"cache_hit":59392,"estimated_new_tokens":11862,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":16734,"ongoing_decode_tokens":0,"pending_prefill_tokens":7518,"num_requests":1,"active_p_offloads":0,"cached_blocks":1462,"cache_hit":0,"new_prefill":71254,"score_linear":16734.0,"score_lmetric":78772},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":34317,"ongoing_decode_tokens":15283,"pending_prefill_tokens":19034,"num_requests":2,"active_p_offloads":0,"cached_blocks":1293,"cache_hit":0,"new_prefill":71254,"score_linear":34317.0,"score_lmetric":180576},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":102368,"ongoing_decode_tokens":102368,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1917,"cache_hit":59392,"new_prefill":11862,"score_linear":42976.0,"score_lmetric":23724},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":69465,"ongoing_decode_tokens":0,"pending_prefill_tokens":3417,"num_requests":1,"active_p_offloads":0,"cached_blocks":1887,"cache_hit":0,"new_prefill":71254,"score_linear":69465.0,"score_lmetric":74671},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":0,"pending_prefill_tokens":164955,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":71254,"score_linear":296027.0,"score_lmetric":708627},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":64229,"ongoing_decode_tokens":64229,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1353,"cache_hit":0,"new_prefill":71254,"score_linear":64229.0,"score_lmetric":142508},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":127253,"ongoing_decode_tokens":20948,"pending_prefill_tokens":44865,"num_requests":2,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":71254,"score_linear":127253.0,"score_lmetric":232238},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":145668,"ongoing_decode_tokens":0,"pending_prefill_tokens":105220,"num_requests":2,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":71254,"score_linear":145668.0,"score_lmetric":352948}],"chosen_score_linear":42976.0,"chosen_score_lmetric":23724,"t_first_token":348829.314386422,"t_first_token_unix":1779779694.3283305,"t_done":348833.882191762,"t_done_unix":1779779698.8961353},{"request_id":"1340290:5:1386504:688","session_id":"1340290","input_length":27220,"t_proxy_recv":348833.176856657,"t_decision_unix":1779779698.1907983,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":26112,"affinity_cache_ratio":0.9592946362968405,"affinity_num_requests":2,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":26112,"estimated_new_tokens":1108,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1476,"cache_hit":0,"new_prefill":27220,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1400,"cache_hit":0,"new_prefill":27220,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":152401,"ongoing_decode_tokens":152401,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1927,"cache_hit":26112,"new_prefill":1108,"score_linear":126289.0,"score_lmetric":2216},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":69465,"ongoing_decode_tokens":0,"pending_prefill_tokens":3417,"num_requests":1,"active_p_offloads":0,"cached_blocks":1887,"cache_hit":0,"new_prefill":27220,"score_linear":69465.0,"score_lmetric":30637},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":0,"pending_prefill_tokens":164955,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":27220,"score_linear":296027.0,"score_lmetric":576525},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":50793,"ongoing_decode_tokens":0,"pending_prefill_tokens":22633,"num_requests":1,"active_p_offloads":0,"cached_blocks":1491,"cache_hit":0,"new_prefill":27220,"score_linear":50793.0,"score_lmetric":49853},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":127253,"ongoing_decode_tokens":20948,"pending_prefill_tokens":44865,"num_requests":2,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":1024,"new_prefill":26196,"score_linear":126229.0,"score_lmetric":142122},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":145668,"ongoing_decode_tokens":145668,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":27220,"score_linear":145668.0,"score_lmetric":54440}],"chosen_score_linear":126289.0,"chosen_score_lmetric":2216,"t_first_token":348833.474688289,"t_first_token_unix":1779779698.4886315,"t_done":348833.913617185,"t_done_unix":1779779698.92756},{"request_id":"1366128:12:1386045:687","session_id":"1366128","input_length":71578,"t_proxy_recv":348833.898109321,"t_decision_unix":1779779698.912051,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":71168,"affinity_cache_ratio":0.9942719830115399,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":71168,"estimated_new_tokens":410,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1476,"cache_hit":0,"new_prefill":71578,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1400,"cache_hit":0,"new_prefill":71578,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":27220,"ongoing_decode_tokens":27220,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":71168,"new_prefill":410,"score_linear":-43948.0,"score_lmetric":410},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":69465,"ongoing_decode_tokens":0,"pending_prefill_tokens":3417,"num_requests":1,"active_p_offloads":0,"cached_blocks":1887,"cache_hit":0,"new_prefill":71578,"score_linear":69465.0,"score_lmetric":74995},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":0,"pending_prefill_tokens":164955,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":71578,"score_linear":296027.0,"score_lmetric":709599},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":50793,"ongoing_decode_tokens":0,"pending_prefill_tokens":22633,"num_requests":1,"active_p_offloads":0,"cached_blocks":1491,"cache_hit":0,"new_prefill":71578,"score_linear":50793.0,"score_lmetric":94211},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":127253,"ongoing_decode_tokens":20948,"pending_prefill_tokens":44865,"num_requests":2,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":71578,"score_linear":127253.0,"score_lmetric":232886},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":145668,"ongoing_decode_tokens":145668,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":71578,"score_linear":145668.0,"score_lmetric":143156}],"chosen_score_linear":-43948.0,"chosen_score_lmetric":410,"t_first_token":348834.129085992,"t_first_token_unix":1779779699.1430292,"t_done":348834.753288241,"t_done_unix":1779779699.7672317},{"request_id":"1366128:13:1387308:692","session_id":"1366128","input_length":71708,"t_proxy_recv":348834.765032678,"t_decision_unix":1779779699.7789745,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":71168,"affinity_cache_ratio":0.9924694594745356,"affinity_num_requests":1,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":71168,"estimated_new_tokens":540,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":26837,"ongoing_decode_tokens":26837,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1476,"cache_hit":0,"new_prefill":71708,"score_linear":26837.0,"score_lmetric":71708},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1400,"cache_hit":0,"new_prefill":71708,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":27285,"ongoing_decode_tokens":27285,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1955,"cache_hit":71168,"new_prefill":540,"score_linear":-43883.0,"score_lmetric":540},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":69465,"ongoing_decode_tokens":0,"pending_prefill_tokens":3417,"num_requests":1,"active_p_offloads":0,"cached_blocks":1887,"cache_hit":0,"new_prefill":71708,"score_linear":69465.0,"score_lmetric":75125},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":126340,"pending_prefill_tokens":38615,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":71708,"score_linear":296027.0,"score_lmetric":330969},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":50793,"ongoing_decode_tokens":0,"pending_prefill_tokens":22633,"num_requests":1,"active_p_offloads":0,"cached_blocks":1491,"cache_hit":0,"new_prefill":71708,"score_linear":50793.0,"score_lmetric":94341},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":127253,"ongoing_decode_tokens":20948,"pending_prefill_tokens":44865,"num_requests":2,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":71708,"score_linear":127253.0,"score_lmetric":233146},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":145668,"ongoing_decode_tokens":145668,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":71708,"score_linear":145668.0,"score_lmetric":143416}],"chosen_score_linear":-43883.0,"chosen_score_lmetric":540,"t_first_token":348834.930237059,"t_first_token_unix":1779779699.9441805,"t_done":348835.404048046,"t_done_unix":1779779700.4179916},{"request_id":"1340290:6:1387365:695","session_id":"1340290","input_length":27285,"t_proxy_recv":348833.920811999,"t_decision_unix":1779779698.9347534,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":27136,"affinity_cache_ratio":0.9945391240608393,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":27136,"estimated_new_tokens":149,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1476,"cache_hit":0,"new_prefill":27285,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1400,"cache_hit":0,"new_prefill":27285,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":71578,"ongoing_decode_tokens":0,"pending_prefill_tokens":410,"num_requests":1,"active_p_offloads":0,"cached_blocks":1955,"cache_hit":27136,"new_prefill":149,"score_linear":44442.0,"score_lmetric":559},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":69465,"ongoing_decode_tokens":0,"pending_prefill_tokens":3417,"num_requests":1,"active_p_offloads":0,"cached_blocks":1887,"cache_hit":0,"new_prefill":27285,"score_linear":69465.0,"score_lmetric":30702},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":0,"pending_prefill_tokens":164955,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":27285,"score_linear":296027.0,"score_lmetric":576720},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":50793,"ongoing_decode_tokens":0,"pending_prefill_tokens":22633,"num_requests":1,"active_p_offloads":0,"cached_blocks":1491,"cache_hit":0,"new_prefill":27285,"score_linear":50793.0,"score_lmetric":49918},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":127253,"ongoing_decode_tokens":20948,"pending_prefill_tokens":44865,"num_requests":2,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":1024,"new_prefill":26261,"score_linear":126229.0,"score_lmetric":142252},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":145668,"ongoing_decode_tokens":145668,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":27285,"score_linear":145668.0,"score_lmetric":54570}],"chosen_score_linear":44442.0,"chosen_score_lmetric":559,"t_first_token":348834.161409981,"t_first_token_unix":1779779699.175353,"t_done":348835.830418335,"t_done_unix":1779779700.8443618},{"request_id":"1355484:11:1388276:696","session_id":"1355484","input_length":26837,"t_proxy_recv":348834.291829596,"t_decision_unix":1779779699.305771,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":25088,"affinity_cache_ratio":0.9348287811603383,"affinity_num_requests":0,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":25088,"estimated_new_tokens":1749,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1476,"cache_hit":25088,"new_prefill":1749,"score_linear":-25088.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1400,"cache_hit":0,"new_prefill":26837,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":98863,"ongoing_decode_tokens":98863,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1955,"cache_hit":0,"new_prefill":26837,"score_linear":98863.0,"score_lmetric":53674},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":69465,"ongoing_decode_tokens":0,"pending_prefill_tokens":3417,"num_requests":1,"active_p_offloads":0,"cached_blocks":1887,"cache_hit":0,"new_prefill":26837,"score_linear":69465.0,"score_lmetric":30254},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":0,"pending_prefill_tokens":164955,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":26837,"score_linear":296027.0,"score_lmetric":575376},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":50793,"ongoing_decode_tokens":0,"pending_prefill_tokens":22633,"num_requests":1,"active_p_offloads":0,"cached_blocks":1491,"cache_hit":0,"new_prefill":26837,"score_linear":50793.0,"score_lmetric":49470},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":127253,"ongoing_decode_tokens":20948,"pending_prefill_tokens":44865,"num_requests":2,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":12800,"new_prefill":14037,"score_linear":114453.0,"score_lmetric":117804},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":145668,"ongoing_decode_tokens":145668,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":12800,"new_prefill":14037,"score_linear":132868.0,"score_lmetric":28074}],"chosen_score_linear":-25088.0,"chosen_score_lmetric":0,"t_first_token":348834.63264405,"t_first_token_unix":1779779699.646588,"t_done":348835.875979177,"t_done_unix":1779779700.8899226},{"request_id":"1382804:2:1388620:698","session_id":"1382804","input_length":3678,"t_proxy_recv":348835.667784804,"t_decision_unix":1779779700.681726,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":512,"affinity_cache_ratio":0.13920609026644915,"affinity_num_requests":1,"avg_num_requests":1.5,"fallback_score":3166,"tie_break_used":false,"cache_hit":512,"estimated_new_tokens":3166,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":26837,"ongoing_decode_tokens":26837,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1476,"cache_hit":512,"new_prefill":3166,"score_linear":26325.0,"score_lmetric":3166},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":71899,"ongoing_decode_tokens":0,"pending_prefill_tokens":71899,"num_requests":1,"active_p_offloads":0,"cached_blocks":1400,"cache_hit":0,"new_prefill":3678,"score_linear":71899.0,"score_lmetric":75577},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":27285,"ongoing_decode_tokens":27285,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1956,"cache_hit":0,"new_prefill":3678,"score_linear":27285.0,"score_lmetric":3678},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":69465,"ongoing_decode_tokens":0,"pending_prefill_tokens":3417,"num_requests":1,"active_p_offloads":0,"cached_blocks":1887,"cache_hit":0,"new_prefill":3678,"score_linear":69465.0,"score_lmetric":7095},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":126340,"pending_prefill_tokens":38615,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":3678,"score_linear":296027.0,"score_lmetric":126879},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":50793,"ongoing_decode_tokens":0,"pending_prefill_tokens":22633,"num_requests":1,"active_p_offloads":0,"cached_blocks":1491,"cache_hit":0,"new_prefill":3678,"score_linear":50793.0,"score_lmetric":26311},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":127253,"ongoing_decode_tokens":20948,"pending_prefill_tokens":44865,"num_requests":2,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":3678,"score_linear":127253.0,"score_lmetric":97086},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":145668,"ongoing_decode_tokens":145668,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":3678,"score_linear":145668.0,"score_lmetric":7356}],"chosen_score_linear":26325.0,"chosen_score_lmetric":3166,"t_first_token":348835.875114171,"t_first_token_unix":1779779700.8890576,"t_done":348835.927641685,"t_done_unix":1779779700.9415846},{"request_id":"1342921:3:1384020:670","session_id":"1342921","input_length":69465,"t_proxy_recv":348819.597651794,"t_decision_unix":1779779684.611593,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":66048,"affinity_cache_ratio":0.950809760310948,"affinity_num_requests":0,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":66048,"estimated_new_tokens":3417,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1438,"cache_hit":0,"new_prefill":69465,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1284,"cache_hit":0,"new_prefill":69465,"score_linear":15283.0,"score_lmetric":69465},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":140377,"ongoing_decode_tokens":140377,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1916,"cache_hit":0,"new_prefill":69465,"score_linear":140377.0,"score_lmetric":138930},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1887,"cache_hit":66048,"new_prefill":3417,"score_linear":-66048.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":246162,"ongoing_decode_tokens":0,"pending_prefill_tokens":164754,"num_requests":2,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":69465,"score_linear":246162.0,"score_lmetric":468438},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":78218,"ongoing_decode_tokens":13989,"pending_prefill_tokens":56549,"num_requests":3,"active_p_offloads":0,"cached_blocks":1327,"cache_hit":0,"new_prefill":69465,"score_linear":78218.0,"score_lmetric":378042},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":81806,"ongoing_decode_tokens":81806,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1341,"cache_hit":0,"new_prefill":69465,"score_linear":81806.0,"score_lmetric":138930},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":145668,"ongoing_decode_tokens":0,"pending_prefill_tokens":105220,"num_requests":2,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":69465,"score_linear":145668.0,"score_lmetric":349370}],"chosen_score_linear":-66048.0,"chosen_score_lmetric":0,"t_first_token":348836.356010927,"t_first_token_unix":1779779701.369955,"t_done":348837.342274256,"t_done_unix":1779779702.3562179},{"request_id":"1381387:3:1387093:691","session_id":"1381387","input_length":50793,"t_proxy_recv":348830.170700565,"t_decision_unix":1779779695.1846414,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":28160,"affinity_cache_ratio":0.5544071033410115,"affinity_num_requests":0,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":28160,"estimated_new_tokens":22633,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1476,"cache_hit":0,"new_prefill":50793,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":12399,"ongoing_decode_tokens":0,"pending_prefill_tokens":11375,"num_requests":1,"active_p_offloads":0,"cached_blocks":1365,"cache_hit":0,"new_prefill":50793,"score_linear":12399.0,"score_lmetric":62168},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":173622,"ongoing_decode_tokens":173622,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1917,"cache_hit":0,"new_prefill":50793,"score_linear":173622.0,"score_lmetric":152379},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":69465,"ongoing_decode_tokens":0,"pending_prefill_tokens":3417,"num_requests":1,"active_p_offloads":0,"cached_blocks":1887,"cache_hit":0,"new_prefill":50793,"score_linear":69465.0,"score_lmetric":54210},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":0,"pending_prefill_tokens":164955,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":50793,"score_linear":296027.0,"score_lmetric":647244},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1491,"cache_hit":28160,"new_prefill":22633,"score_linear":-28160.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":127253,"ongoing_decode_tokens":20948,"pending_prefill_tokens":44865,"num_requests":2,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":50793,"score_linear":127253.0,"score_lmetric":191316},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":145668,"ongoing_decode_tokens":98267,"pending_prefill_tokens":19753,"num_requests":2,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":50793,"score_linear":145668.0,"score_lmetric":141092}],"chosen_score_linear":-28160.0,"chosen_score_lmetric":0,"t_first_token":348836.124231526,"t_first_token_unix":1779779701.1381755,"t_done":348838.170773526,"t_done_unix":1779779703.1847196},{"request_id":"1342921:4:1389290:702","session_id":"1342921","input_length":70990,"t_proxy_recv":348838.084188556,"t_decision_unix":1779779703.098129,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":69120,"affinity_cache_ratio":0.9736582617270038,"affinity_num_requests":0,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":69120,"estimated_new_tokens":1870,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1485,"cache_hit":0,"new_prefill":70990,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":92082,"ongoing_decode_tokens":0,"pending_prefill_tokens":73138,"num_requests":2,"active_p_offloads":0,"cached_blocks":1400,"cache_hit":0,"new_prefill":70990,"score_linear":92082.0,"score_lmetric":288256},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":83404,"ongoing_decode_tokens":0,"pending_prefill_tokens":11724,"num_requests":1,"active_p_offloads":0,"cached_blocks":1956,"cache_hit":0,"new_prefill":70990,"score_linear":83404.0,"score_lmetric":82714},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1893,"cache_hit":69120,"new_prefill":1870,"score_linear":-69120.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":126340,"pending_prefill_tokens":38615,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":70990,"score_linear":296027.0,"score_lmetric":328815},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":50793,"ongoing_decode_tokens":50793,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1491,"cache_hit":0,"new_prefill":70990,"score_linear":50793.0,"score_lmetric":70990},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":127253,"ongoing_decode_tokens":20948,"pending_prefill_tokens":44865,"num_requests":2,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":70990,"score_linear":127253.0,"score_lmetric":231710},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":145668,"ongoing_decode_tokens":145668,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":70990,"score_linear":145668.0,"score_lmetric":141980}],"chosen_score_linear":-69120.0,"chosen_score_lmetric":0,"t_first_token":348838.86053377,"t_first_token_unix":1779779703.8744779,"t_done":348841.272264647,"t_done_unix":1779779706.2862082},{"request_id":"1390240:1:1390240:705","session_id":"1390240","input_length":5110,"t_proxy_recv":348841.604377841,"t_decision_unix":1779779706.6183188,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":5110,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1485,"cache_hit":0,"new_prefill":5110,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":92082,"ongoing_decode_tokens":0,"pending_prefill_tokens":73138,"num_requests":2,"active_p_offloads":0,"cached_blocks":1400,"cache_hit":0,"new_prefill":5110,"score_linear":92082.0,"score_lmetric":156496},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":121646,"ongoing_decode_tokens":0,"pending_prefill_tokens":12078,"num_requests":2,"active_p_offloads":0,"cached_blocks":1956,"cache_hit":0,"new_prefill":5110,"score_linear":121646.0,"score_lmetric":34376},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":5110,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":126340,"pending_prefill_tokens":38615,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":5110,"score_linear":296027.0,"score_lmetric":131175},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1535,"cache_hit":0,"new_prefill":5110,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":127253,"ongoing_decode_tokens":20948,"pending_prefill_tokens":44865,"num_requests":2,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":5110,"score_linear":127253.0,"score_lmetric":99950},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":145668,"ongoing_decode_tokens":145668,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":5110,"score_linear":145668.0,"score_lmetric":10220}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348841.93060087,"t_first_token_unix":1779779706.9445443,"t_done":348842.206168592,"t_done_unix":1779779707.2201116},{"request_id":"1364090:3:1381959:653","session_id":"1364090","input_length":47401,"t_proxy_recv":348812.418444182,"t_decision_unix":1779779677.4323854,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":27648,"affinity_cache_ratio":0.5832788337798781,"affinity_num_requests":1,"avg_num_requests":2.0,"fallback_score":null,"tie_break_used":false,"cache_hit":27648,"estimated_new_tokens":19753,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":35888,"ongoing_decode_tokens":35888,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1400,"cache_hit":7680,"new_prefill":39721,"score_linear":28208.0,"score_lmetric":39721},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1268,"cache_hit":0,"new_prefill":47401,"score_linear":15283.0,"score_lmetric":47401},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":186692,"ongoing_decode_tokens":117536,"pending_prefill_tokens":12324,"num_requests":4,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":47401,"score_linear":186692.0,"score_lmetric":238900},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":111245,"ongoing_decode_tokens":0,"pending_prefill_tokens":11917,"num_requests":4,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":47401,"score_linear":111245.0,"score_lmetric":237272},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":246162,"ongoing_decode_tokens":0,"pending_prefill_tokens":164754,"num_requests":2,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":7680,"new_prefill":39721,"score_linear":238482.0,"score_lmetric":408950},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":13989,"ongoing_decode_tokens":13989,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1327,"cache_hit":0,"new_prefill":47401,"score_linear":13989.0,"score_lmetric":47401},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":126629,"ongoing_decode_tokens":126629,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1250,"cache_hit":7680,"new_prefill":39721,"score_linear":118949.0,"score_lmetric":79442},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":98267,"ongoing_decode_tokens":0,"pending_prefill_tokens":85467,"num_requests":1,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":27648,"new_prefill":19753,"score_linear":70619.0,"score_lmetric":105220}],"chosen_score_linear":70619.0,"chosen_score_lmetric":105220,"t_first_token":348831.358869943,"t_first_token_unix":1779779696.3728132,"t_done":348842.96799638,"t_done_unix":1779779707.98194},{"request_id":"1390356:1:1390356:706","session_id":"1390356","input_length":1349,"t_proxy_recv":348842.059657258,"t_decision_unix":1779779707.0735986,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.5,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":1349,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":5110,"ongoing_decode_tokens":5110,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1485,"cache_hit":0,"new_prefill":1349,"score_linear":5110.0,"score_lmetric":1349},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":92082,"ongoing_decode_tokens":0,"pending_prefill_tokens":73138,"num_requests":2,"active_p_offloads":0,"cached_blocks":1400,"cache_hit":0,"new_prefill":1349,"score_linear":92082.0,"score_lmetric":148974},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":121646,"ongoing_decode_tokens":0,"pending_prefill_tokens":12078,"num_requests":2,"active_p_offloads":0,"cached_blocks":1956,"cache_hit":0,"new_prefill":1349,"score_linear":121646.0,"score_lmetric":26854},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":1349,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":126340,"pending_prefill_tokens":38615,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":1349,"score_linear":296027.0,"score_lmetric":119892},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1535,"cache_hit":0,"new_prefill":1349,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":127253,"ongoing_decode_tokens":20948,"pending_prefill_tokens":44865,"num_requests":2,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":1349,"score_linear":127253.0,"score_lmetric":92428},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":145668,"ongoing_decode_tokens":145668,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":1349,"score_linear":145668.0,"score_lmetric":2698}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348842.155127433,"t_first_token_unix":1779779707.1690705,"t_done":348843.844400682,"t_done_unix":1779779708.8583438},{"request_id":"1355951:4:1389858:703","session_id":"1355951","input_length":38242,"t_proxy_recv":348840.209188734,"t_decision_unix":1779779705.2231295,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":37888,"affinity_cache_ratio":0.9907431619685163,"affinity_num_requests":1,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":37888,"estimated_new_tokens":354,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1485,"cache_hit":0,"new_prefill":38242,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":92082,"ongoing_decode_tokens":0,"pending_prefill_tokens":73138,"num_requests":2,"active_p_offloads":0,"cached_blocks":1400,"cache_hit":0,"new_prefill":38242,"score_linear":92082.0,"score_lmetric":222760},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":83404,"ongoing_decode_tokens":0,"pending_prefill_tokens":11724,"num_requests":1,"active_p_offloads":0,"cached_blocks":1956,"cache_hit":37888,"new_prefill":354,"score_linear":45516.0,"score_lmetric":12078},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":70990,"ongoing_decode_tokens":70990,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1893,"cache_hit":0,"new_prefill":38242,"score_linear":70990.0,"score_lmetric":38242},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":126340,"pending_prefill_tokens":38615,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":38242,"score_linear":296027.0,"score_lmetric":230571},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1535,"cache_hit":0,"new_prefill":38242,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":127253,"ongoing_decode_tokens":20948,"pending_prefill_tokens":44865,"num_requests":2,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":38242,"score_linear":127253.0,"score_lmetric":166214},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":145668,"ongoing_decode_tokens":145668,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":38242,"score_linear":145668.0,"score_lmetric":76484}],"chosen_score_linear":45516.0,"chosen_score_lmetric":12078,"t_first_token":348843.514759028,"t_first_token_unix":1779779708.5287023,"t_done":348844.032481447,"t_done_unix":1779779709.046425},{"request_id":"1366128:14:1389156:701","session_id":"1366128","input_length":83404,"t_proxy_recv":348837.663741385,"t_decision_unix":1779779702.6776829,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":71680,"affinity_cache_ratio":0.8594312023404154,"affinity_num_requests":0,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":71680,"estimated_new_tokens":11724,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1485,"cache_hit":0,"new_prefill":83404,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":92082,"ongoing_decode_tokens":0,"pending_prefill_tokens":73138,"num_requests":2,"active_p_offloads":0,"cached_blocks":1400,"cache_hit":0,"new_prefill":83404,"score_linear":92082.0,"score_lmetric":313084},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1956,"cache_hit":71680,"new_prefill":11724,"score_linear":-71680.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1893,"cache_hit":0,"new_prefill":83404,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":126340,"pending_prefill_tokens":38615,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":83404,"score_linear":296027.0,"score_lmetric":366057},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":50793,"ongoing_decode_tokens":50793,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1491,"cache_hit":0,"new_prefill":83404,"score_linear":50793.0,"score_lmetric":83404},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":127253,"ongoing_decode_tokens":20948,"pending_prefill_tokens":44865,"num_requests":2,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":83404,"score_linear":127253.0,"score_lmetric":256538},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":145668,"ongoing_decode_tokens":145668,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":83404,"score_linear":145668.0,"score_lmetric":166808}],"chosen_score_linear":-71680.0,"chosen_score_lmetric":0,"t_first_token":348843.454892859,"t_first_token_unix":1779779708.4688363,"t_done":348844.575651815,"t_done_unix":1779779709.5895958},{"request_id":"1366128:15:1390663:709","session_id":"1366128","input_length":83622,"t_proxy_recv":348844.588806419,"t_decision_unix":1779779709.6027482,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":83456,"affinity_cache_ratio":0.9980148764679151,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":83456,"estimated_new_tokens":166,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1495,"cache_hit":0,"new_prefill":83622,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":92082,"ongoing_decode_tokens":0,"pending_prefill_tokens":73138,"num_requests":2,"active_p_offloads":0,"cached_blocks":1400,"cache_hit":0,"new_prefill":83622,"score_linear":92082.0,"score_lmetric":313520},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1979,"cache_hit":83456,"new_prefill":166,"score_linear":-83456.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":83622,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":182626,"pending_prefill_tokens":761,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":83622,"score_linear":296027.0,"score_lmetric":253149},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1538,"cache_hit":0,"new_prefill":83622,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":127253,"ongoing_decode_tokens":127253,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":83622,"score_linear":127253.0,"score_lmetric":167244},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":98267,"ongoing_decode_tokens":98267,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1117,"cache_hit":0,"new_prefill":83622,"score_linear":98267.0,"score_lmetric":83622}],"chosen_score_linear":-83456.0,"chosen_score_lmetric":0,"t_first_token":348844.805595435,"t_first_token_unix":1779779709.8195395,"t_done":348845.89163413,"t_done_unix":1779779710.905578},{"request_id":"1313181:6:1324908:369","session_id":"1313181","input_length":106305,"t_proxy_recv":348821.074350834,"t_decision_unix":1779779686.0882924,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":61440,"affinity_cache_ratio":0.5779596444193594,"affinity_num_requests":1,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"cache_hit":61440,"estimated_new_tokens":44865,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":10139,"ongoing_decode_tokens":10139,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1438,"cache_hit":24064,"new_prefill":82241,"score_linear":-13925.0,"score_lmetric":164482},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1284,"cache_hit":0,"new_prefill":106305,"score_linear":15283.0,"score_lmetric":106305},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":161742,"ongoing_decode_tokens":140521,"pending_prefill_tokens":229,"num_requests":3,"active_p_offloads":0,"cached_blocks":1916,"cache_hit":0,"new_prefill":106305,"score_linear":161742.0,"score_lmetric":319602},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":69465,"ongoing_decode_tokens":0,"pending_prefill_tokens":3417,"num_requests":1,"active_p_offloads":0,"cached_blocks":1887,"cache_hit":0,"new_prefill":106305,"score_linear":69465.0,"score_lmetric":109722},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":246162,"ongoing_decode_tokens":0,"pending_prefill_tokens":164754,"num_requests":2,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":106305,"score_linear":246162.0,"score_lmetric":542118},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":78218,"ongoing_decode_tokens":13989,"pending_prefill_tokens":56549,"num_requests":3,"active_p_offloads":0,"cached_blocks":1327,"cache_hit":0,"new_prefill":106305,"score_linear":78218.0,"score_lmetric":488562},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":20948,"ongoing_decode_tokens":20948,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":61440,"new_prefill":44865,"score_linear":-40492.0,"score_lmetric":44865},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":145668,"ongoing_decode_tokens":0,"pending_prefill_tokens":105220,"num_requests":2,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":12800,"new_prefill":93505,"score_linear":132868.0,"score_lmetric":397450}],"chosen_score_linear":-40492.0,"chosen_score_lmetric":44865,"t_first_token":348844.484483868,"t_first_token_unix":1779779709.4984279,"t_done":348846.847207278,"t_done_unix":1779779711.8611512},{"request_id":"1391584:1:1391584:713","session_id":"1391584","input_length":15302,"t_proxy_recv":348846.124970746,"t_decision_unix":1779779711.138912,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":15302,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1495,"cache_hit":0,"new_prefill":15302,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":92082,"ongoing_decode_tokens":0,"pending_prefill_tokens":73138,"num_requests":2,"active_p_offloads":0,"cached_blocks":1400,"cache_hit":0,"new_prefill":15302,"score_linear":92082.0,"score_lmetric":176880},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1979,"cache_hit":0,"new_prefill":15302,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":15302,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":182626,"pending_prefill_tokens":761,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":15302,"score_linear":296027.0,"score_lmetric":48189},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":28307,"ongoing_decode_tokens":0,"pending_prefill_tokens":25747,"num_requests":1,"active_p_offloads":0,"cached_blocks":1538,"cache_hit":0,"new_prefill":15302,"score_linear":28307.0,"score_lmetric":41049},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":127253,"ongoing_decode_tokens":127253,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":15302,"score_linear":127253.0,"score_lmetric":30604},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":98267,"ongoing_decode_tokens":98267,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1117,"cache_hit":0,"new_prefill":15302,"score_linear":98267.0,"score_lmetric":15302}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348847.486834224,"t_first_token_unix":1779779712.5007777,"t_done":348847.703247768,"t_done_unix":1779779712.7171912},{"request_id":"1363943:8:1379215:635","session_id":"1363943","input_length":20948,"t_proxy_recv":348813.737020836,"t_decision_unix":1779779678.750962,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":20480,"affinity_cache_ratio":0.97765896505633,"affinity_num_requests":1,"avg_num_requests":2.375,"fallback_score":null,"tie_break_used":false,"cache_hit":20480,"estimated_new_tokens":468,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":35888,"ongoing_decode_tokens":35888,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1400,"cache_hit":8704,"new_prefill":12244,"score_linear":27184.0,"score_lmetric":12244},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":18317,"ongoing_decode_tokens":15283,"pending_prefill_tokens":3034,"num_requests":2,"active_p_offloads":0,"cached_blocks":1268,"cache_hit":0,"new_prefill":20948,"score_linear":18317.0,"score_lmetric":47964},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":224803,"ongoing_decode_tokens":186692,"pending_prefill_tokens":3295,"num_requests":5,"active_p_offloads":0,"cached_blocks":1731,"cache_hit":0,"new_prefill":20948,"score_linear":224803.0,"score_lmetric":121215},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":111245,"ongoing_decode_tokens":0,"pending_prefill_tokens":11917,"num_requests":4,"active_p_offloads":0,"cached_blocks":1866,"cache_hit":0,"new_prefill":20948,"score_linear":111245.0,"score_lmetric":131460},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":246162,"ongoing_decode_tokens":0,"pending_prefill_tokens":164754,"num_requests":2,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":4096,"new_prefill":16852,"score_linear":242066.0,"score_lmetric":363212},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":68001,"ongoing_decode_tokens":13989,"pending_prefill_tokens":54012,"num_requests":2,"active_p_offloads":0,"cached_blocks":1327,"cache_hit":0,"new_prefill":20948,"score_linear":68001.0,"score_lmetric":149920},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":105902,"ongoing_decode_tokens":105902,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1253,"cache_hit":20480,"new_prefill":468,"score_linear":85422.0,"score_lmetric":468},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":145668,"ongoing_decode_tokens":0,"pending_prefill_tokens":105220,"num_requests":2,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":4096,"new_prefill":16852,"score_linear":141572.0,"score_lmetric":244144}],"chosen_score_linear":85422.0,"chosen_score_lmetric":468,"t_first_token":348813.81263641,"t_first_token_unix":1779779678.8265798,"t_done":348848.702845104,"t_done_unix":1779779713.7167892},{"request_id":"1355484:12:1391879:714","session_id":"1355484","input_length":28382,"t_proxy_recv":348847.254326527,"t_decision_unix":1779779712.2682679,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":26624,"affinity_cache_ratio":0.9380593333803114,"affinity_num_requests":0,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":26624,"estimated_new_tokens":1758,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1495,"cache_hit":26624,"new_prefill":1758,"score_linear":-26624.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":92082,"ongoing_decode_tokens":0,"pending_prefill_tokens":73138,"num_requests":2,"active_p_offloads":0,"cached_blocks":1400,"cache_hit":0,"new_prefill":28382,"score_linear":92082.0,"score_lmetric":203040},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1979,"cache_hit":0,"new_prefill":28382,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15302,"ongoing_decode_tokens":0,"pending_prefill_tokens":15302,"num_requests":1,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":28382,"score_linear":15302.0,"score_lmetric":43684},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":182626,"pending_prefill_tokens":761,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":28382,"score_linear":296027.0,"score_lmetric":87429},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":28307,"ongoing_decode_tokens":0,"pending_prefill_tokens":25747,"num_requests":1,"active_p_offloads":0,"cached_blocks":1538,"cache_hit":0,"new_prefill":28382,"score_linear":28307.0,"score_lmetric":54129},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":82491,"ongoing_decode_tokens":82491,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1345,"cache_hit":12800,"new_prefill":15582,"score_linear":69691.0,"score_lmetric":31164},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":98267,"ongoing_decode_tokens":98267,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1117,"cache_hit":12800,"new_prefill":15582,"score_linear":85467.0,"score_lmetric":15582}],"chosen_score_linear":-26624.0,"chosen_score_lmetric":0,"t_first_token":348847.613081134,"t_first_token_unix":1779779712.6270244,"t_done":348849.250573655,"t_done_unix":1779779714.2645178},{"request_id":"1391584:2:1392466:715","session_id":"1391584","input_length":15483,"t_proxy_recv":348849.435419539,"t_decision_unix":1779779714.4493608,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":14848,"affinity_cache_ratio":0.958987276367629,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":14848,"estimated_new_tokens":635,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1498,"cache_hit":0,"new_prefill":15483,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":92082,"ongoing_decode_tokens":0,"pending_prefill_tokens":73138,"num_requests":2,"active_p_offloads":0,"cached_blocks":1400,"cache_hit":0,"new_prefill":15483,"score_linear":92082.0,"score_lmetric":177242},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1979,"cache_hit":0,"new_prefill":15483,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1926,"cache_hit":14848,"new_prefill":635,"score_linear":-14848.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":182626,"pending_prefill_tokens":761,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":15483,"score_linear":296027.0,"score_lmetric":48732},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":28307,"ongoing_decode_tokens":28307,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1538,"cache_hit":0,"new_prefill":15483,"score_linear":28307.0,"score_lmetric":15483},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":85184,"ongoing_decode_tokens":85184,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1347,"cache_hit":0,"new_prefill":15483,"score_linear":85184.0,"score_lmetric":30966},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":98267,"ongoing_decode_tokens":98267,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1117,"cache_hit":0,"new_prefill":15483,"score_linear":98267.0,"score_lmetric":15483}],"chosen_score_linear":-14848.0,"chosen_score_lmetric":0,"t_first_token":348849.512089563,"t_first_token_unix":1779779714.5260327,"t_done":348849.899542798,"t_done_unix":1779779714.9134858},{"request_id":"1391404:1:1391404:712","session_id":"1391404","input_length":28307,"t_proxy_recv":348845.603549192,"t_decision_unix":1779779710.6174905,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":5,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":false,"cache_hit":2560,"estimated_new_tokens":25747,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1495,"cache_hit":0,"new_prefill":28307,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":92082,"ongoing_decode_tokens":0,"pending_prefill_tokens":73138,"num_requests":2,"active_p_offloads":0,"cached_blocks":1400,"cache_hit":0,"new_prefill":28307,"score_linear":92082.0,"score_lmetric":202890},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":83622,"ongoing_decode_tokens":83622,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1979,"cache_hit":0,"new_prefill":28307,"score_linear":83622.0,"score_lmetric":28307},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":28307,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":182626,"pending_prefill_tokens":761,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":28307,"score_linear":296027.0,"score_lmetric":87204},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1538,"cache_hit":2560,"new_prefill":25747,"score_linear":-2560.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":127253,"ongoing_decode_tokens":127253,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":28307,"score_linear":127253.0,"score_lmetric":56614},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":98267,"ongoing_decode_tokens":98267,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1117,"cache_hit":0,"new_prefill":28307,"score_linear":98267.0,"score_lmetric":28307}],"chosen_score_linear":-2560.0,"chosen_score_lmetric":0,"t_first_token":348849.038883794,"t_first_token_unix":1779779714.0528274,"t_done":348850.362768995,"t_done_unix":1779779715.376712},{"request_id":"1390240:2:1392954:717","session_id":"1390240","input_length":7440,"t_proxy_recv":348851.152203951,"t_decision_unix":1779779716.1661453,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":5120,"affinity_cache_ratio":0.6881720430107527,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":5120,"estimated_new_tokens":2320,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1498,"cache_hit":5120,"new_prefill":2320,"score_linear":-5120.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":92082,"ongoing_decode_tokens":0,"pending_prefill_tokens":73138,"num_requests":2,"active_p_offloads":0,"cached_blocks":1400,"cache_hit":0,"new_prefill":7440,"score_linear":92082.0,"score_lmetric":161156},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":84897,"ongoing_decode_tokens":84897,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1979,"cache_hit":0,"new_prefill":7440,"score_linear":84897.0,"score_lmetric":7440},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1927,"cache_hit":0,"new_prefill":7440,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":182626,"pending_prefill_tokens":761,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":7440,"score_linear":296027.0,"score_lmetric":24603},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":7440,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":85184,"ongoing_decode_tokens":85184,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1347,"cache_hit":0,"new_prefill":7440,"score_linear":85184.0,"score_lmetric":14880},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":98267,"ongoing_decode_tokens":98267,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1117,"cache_hit":0,"new_prefill":7440,"score_linear":98267.0,"score_lmetric":7440}],"chosen_score_linear":-5120.0,"chosen_score_lmetric":0,"t_first_token":348851.376930127,"t_first_token_unix":1779779716.3908732,"t_done":348851.604100707,"t_done_unix":1779779716.6180444},{"request_id":"1313181:6:1379623:639","session_id":"1313181","input_length":61543,"t_proxy_recv":348846.856172822,"t_decision_unix":1779779711.8701143,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":61440,"affinity_cache_ratio":0.9983263734299596,"affinity_num_requests":1,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":61440,"estimated_new_tokens":103,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1495,"cache_hit":24064,"new_prefill":37479,"score_linear":-24064.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":92082,"ongoing_decode_tokens":0,"pending_prefill_tokens":73138,"num_requests":2,"active_p_offloads":0,"cached_blocks":1400,"cache_hit":0,"new_prefill":61543,"score_linear":92082.0,"score_lmetric":269362},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1979,"cache_hit":0,"new_prefill":61543,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":15302,"ongoing_decode_tokens":0,"pending_prefill_tokens":15302,"num_requests":1,"active_p_offloads":0,"cached_blocks":1897,"cache_hit":0,"new_prefill":61543,"score_linear":15302.0,"score_lmetric":76845},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":182626,"pending_prefill_tokens":761,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":61543,"score_linear":296027.0,"score_lmetric":186912},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":28307,"ongoing_decode_tokens":0,"pending_prefill_tokens":25747,"num_requests":1,"active_p_offloads":0,"cached_blocks":1538,"cache_hit":0,"new_prefill":61543,"score_linear":28307.0,"score_lmetric":87290},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":20948,"ongoing_decode_tokens":20948,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1345,"cache_hit":61440,"new_prefill":103,"score_linear":-40492.0,"score_lmetric":103},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":98267,"ongoing_decode_tokens":98267,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1117,"cache_hit":12800,"new_prefill":48743,"score_linear":85467.0,"score_lmetric":48743}],"chosen_score_linear":-40492.0,"chosen_score_lmetric":103,"t_first_token":348846.943220641,"t_first_token_unix":1779779711.957164,"t_done":348851.762056982,"t_done_unix":1779779716.7760007},{"request_id":"1366128:16:1392704:716","session_id":"1366128","input_length":84897,"t_proxy_recv":348850.336148423,"t_decision_unix":1779779715.3500893,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":83456,"affinity_cache_ratio":0.9830264909242965,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":83456,"estimated_new_tokens":1441,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1498,"cache_hit":0,"new_prefill":84897,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":92082,"ongoing_decode_tokens":0,"pending_prefill_tokens":73138,"num_requests":2,"active_p_offloads":0,"cached_blocks":1400,"cache_hit":0,"new_prefill":84897,"score_linear":92082.0,"score_lmetric":316070},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1979,"cache_hit":83456,"new_prefill":1441,"score_linear":-83456.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1927,"cache_hit":0,"new_prefill":84897,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":182626,"pending_prefill_tokens":761,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":84897,"score_linear":296027.0,"score_lmetric":256974},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":28307,"ongoing_decode_tokens":28307,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1538,"cache_hit":0,"new_prefill":84897,"score_linear":28307.0,"score_lmetric":84897},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":85184,"ongoing_decode_tokens":85184,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1347,"cache_hit":0,"new_prefill":84897,"score_linear":85184.0,"score_lmetric":169794},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":98267,"ongoing_decode_tokens":98267,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1117,"cache_hit":0,"new_prefill":84897,"score_linear":98267.0,"score_lmetric":84897}],"chosen_score_linear":-83456.0,"chosen_score_lmetric":0,"t_first_token":348851.106562728,"t_first_token_unix":1779779716.120506,"t_done":348853.015523537,"t_done_unix":1779779718.0294669},{"request_id":"1388507:1:1388507:697","session_id":"1388507","input_length":71899,"t_proxy_recv":348835.215825193,"t_decision_unix":1779779700.2297664,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.5,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":71899,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":26837,"ongoing_decode_tokens":26837,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1476,"cache_hit":0,"new_prefill":71899,"score_linear":26837.0,"score_lmetric":71899},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1400,"cache_hit":0,"new_prefill":71899,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":98993,"ongoing_decode_tokens":98993,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1955,"cache_hit":0,"new_prefill":71899,"score_linear":98993.0,"score_lmetric":143798},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":69465,"ongoing_decode_tokens":0,"pending_prefill_tokens":3417,"num_requests":1,"active_p_offloads":0,"cached_blocks":1887,"cache_hit":0,"new_prefill":71899,"score_linear":69465.0,"score_lmetric":75316},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":126340,"pending_prefill_tokens":38615,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":71899,"score_linear":296027.0,"score_lmetric":331542},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":50793,"ongoing_decode_tokens":0,"pending_prefill_tokens":22633,"num_requests":1,"active_p_offloads":0,"cached_blocks":1491,"cache_hit":0,"new_prefill":71899,"score_linear":50793.0,"score_lmetric":94532},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":127253,"ongoing_decode_tokens":20948,"pending_prefill_tokens":44865,"num_requests":2,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":71899,"score_linear":127253.0,"score_lmetric":233528},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":145668,"ongoing_decode_tokens":145668,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":71899,"score_linear":145668.0,"score_lmetric":143798}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348853.194890919,"t_first_token_unix":1779779718.2088351,"t_done":348853.613455873,"t_done_unix":1779779718.6273992},{"request_id":"1384947:2:1388842:700","session_id":"1384947","input_length":20183,"t_proxy_recv":348836.428135814,"t_decision_unix":1779779701.4420772,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":18944,"affinity_cache_ratio":0.9386117029182975,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":18944,"estimated_new_tokens":1239,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1485,"cache_hit":0,"new_prefill":20183,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":71899,"ongoing_decode_tokens":0,"pending_prefill_tokens":71899,"num_requests":1,"active_p_offloads":0,"cached_blocks":1400,"cache_hit":18944,"new_prefill":1239,"score_linear":52955.0,"score_lmetric":73138},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1956,"cache_hit":0,"new_prefill":20183,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":69465,"ongoing_decode_tokens":69465,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1887,"cache_hit":0,"new_prefill":20183,"score_linear":69465.0,"score_lmetric":20183},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":126340,"pending_prefill_tokens":38615,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":20183,"score_linear":296027.0,"score_lmetric":176394},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":50793,"ongoing_decode_tokens":50793,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1491,"cache_hit":0,"new_prefill":20183,"score_linear":50793.0,"score_lmetric":20183},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":127253,"ongoing_decode_tokens":20948,"pending_prefill_tokens":44865,"num_requests":2,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":20183,"score_linear":127253.0,"score_lmetric":130096},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":145668,"ongoing_decode_tokens":145668,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":20183,"score_linear":145668.0,"score_lmetric":40366}],"chosen_score_linear":52955.0,"chosen_score_lmetric":73138,"t_first_token":348853.195866506,"t_first_token_unix":1779779718.2098103,"t_done":348854.426748337,"t_done_unix":1779779719.4406922},{"request_id":"1384947:3:1390898:710","session_id":"1384947","input_length":20683,"t_proxy_recv":348854.435098945,"t_decision_unix":1779779719.4490404,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":19968,"affinity_cache_ratio":0.9654305468258957,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":19968,"estimated_new_tokens":715,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1502,"cache_hit":0,"new_prefill":20683,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1542,"cache_hit":19968,"new_prefill":715,"score_linear":-19968.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1982,"cache_hit":0,"new_prefill":20683,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1927,"cache_hit":0,"new_prefill":20683,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":182626,"pending_prefill_tokens":761,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":20683,"score_linear":296027.0,"score_lmetric":64332},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":101803,"ongoing_decode_tokens":0,"pending_prefill_tokens":28587,"num_requests":1,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":20683,"score_linear":101803.0,"score_lmetric":49270},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":145452,"ongoing_decode_tokens":23641,"pending_prefill_tokens":59859,"num_requests":2,"active_p_offloads":0,"cached_blocks":1348,"cache_hit":0,"new_prefill":20683,"score_linear":145452.0,"score_lmetric":161084},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":147211,"ongoing_decode_tokens":147211,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1117,"cache_hit":0,"new_prefill":20683,"score_linear":147211.0,"score_lmetric":41366}],"chosen_score_linear":-19968.0,"chosen_score_lmetric":0,"t_first_token":348854.56726258,"t_first_token_unix":1779779719.5812058,"t_done":348855.753271625,"t_done_unix":1779779720.7672145},{"request_id":"1394167:1:1394167:721","session_id":"1394167","input_length":4675,"t_proxy_recv":348855.422196083,"t_decision_unix":1779779720.4361374,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":4675,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1502,"cache_hit":0,"new_prefill":4675,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":20683,"ongoing_decode_tokens":20683,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1542,"cache_hit":0,"new_prefill":4675,"score_linear":20683.0,"score_lmetric":4675},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1982,"cache_hit":0,"new_prefill":4675,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1927,"cache_hit":0,"new_prefill":4675,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":182626,"pending_prefill_tokens":761,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":4675,"score_linear":296027.0,"score_lmetric":16308},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":135124,"ongoing_decode_tokens":0,"pending_prefill_tokens":33748,"num_requests":2,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":4675,"score_linear":135124.0,"score_lmetric":76846},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":145452,"ongoing_decode_tokens":23641,"pending_prefill_tokens":59859,"num_requests":2,"active_p_offloads":0,"cached_blocks":1348,"cache_hit":0,"new_prefill":4675,"score_linear":145452.0,"score_lmetric":129068},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":147211,"ongoing_decode_tokens":147211,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1117,"cache_hit":0,"new_prefill":4675,"score_linear":147211.0,"score_lmetric":9350}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348855.71458388,"t_first_token_unix":1779779720.728527,"t_done":348856.091799947,"t_done_unix":1779779721.1057432},{"request_id":"1394428:1:1394428:724","session_id":"1394428","input_length":982,"t_proxy_recv":348856.304159943,"t_decision_unix":1779779721.3181012,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":982,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":28665,"ongoing_decode_tokens":28665,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1511,"cache_hit":0,"new_prefill":982,"score_linear":28665.0,"score_lmetric":982},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1543,"cache_hit":0,"new_prefill":982,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1982,"cache_hit":0,"new_prefill":982,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1927,"cache_hit":0,"new_prefill":982,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":182626,"pending_prefill_tokens":761,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":982,"score_linear":296027.0,"score_lmetric":5229},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":135124,"ongoing_decode_tokens":0,"pending_prefill_tokens":33748,"num_requests":2,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":512,"new_prefill":470,"score_linear":134612.0,"score_lmetric":68436},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":145452,"ongoing_decode_tokens":23641,"pending_prefill_tokens":59859,"num_requests":2,"active_p_offloads":0,"cached_blocks":1348,"cache_hit":0,"new_prefill":982,"score_linear":145452.0,"score_lmetric":121682},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":147211,"ongoing_decode_tokens":147211,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1117,"cache_hit":0,"new_prefill":982,"score_linear":147211.0,"score_lmetric":1964}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348856.389588926,"t_first_token_unix":1779779721.403533,"t_done":348856.393185203,"t_done_unix":1779779721.407128},{"request_id":"1355484:13:1394384:723","session_id":"1355484","input_length":28665,"t_proxy_recv":348856.162675633,"t_decision_unix":1779779721.176617,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":28160,"affinity_cache_ratio":0.9823826966684109,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":28160,"estimated_new_tokens":505,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1511,"cache_hit":28160,"new_prefill":505,"score_linear":-28160.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1543,"cache_hit":0,"new_prefill":28665,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1982,"cache_hit":0,"new_prefill":28665,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1927,"cache_hit":0,"new_prefill":28665,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":182626,"pending_prefill_tokens":761,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":28665,"score_linear":296027.0,"score_lmetric":88278},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":135124,"ongoing_decode_tokens":0,"pending_prefill_tokens":33748,"num_requests":2,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":28665,"score_linear":135124.0,"score_lmetric":124826},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":145452,"ongoing_decode_tokens":23641,"pending_prefill_tokens":59859,"num_requests":2,"active_p_offloads":0,"cached_blocks":1348,"cache_hit":12800,"new_prefill":15865,"score_linear":132652.0,"score_lmetric":151448},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":147211,"ongoing_decode_tokens":147211,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1117,"cache_hit":12800,"new_prefill":15865,"score_linear":134411.0,"score_lmetric":31730}],"chosen_score_linear":-28160.0,"chosen_score_lmetric":0,"t_first_token":348856.221328364,"t_first_token_unix":1779779721.235272,"t_done":348856.66644084,"t_done_unix":1779779721.6803837},{"request_id":"1394432:1:1394432:725","session_id":"1394432","input_length":4736,"t_proxy_recv":348856.323330231,"t_decision_unix":1779779721.3372717,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":4736,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":28665,"ongoing_decode_tokens":28665,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1511,"cache_hit":0,"new_prefill":4736,"score_linear":28665.0,"score_lmetric":4736},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1543,"cache_hit":0,"new_prefill":4736,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":982,"ongoing_decode_tokens":0,"pending_prefill_tokens":982,"num_requests":1,"active_p_offloads":0,"cached_blocks":1982,"cache_hit":0,"new_prefill":4736,"score_linear":982.0,"score_lmetric":5718},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1927,"cache_hit":0,"new_prefill":4736,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":182626,"pending_prefill_tokens":761,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":4736,"score_linear":296027.0,"score_lmetric":16491},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":135124,"ongoing_decode_tokens":0,"pending_prefill_tokens":33748,"num_requests":2,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":4736,"score_linear":135124.0,"score_lmetric":76968},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":145452,"ongoing_decode_tokens":23641,"pending_prefill_tokens":59859,"num_requests":2,"active_p_offloads":0,"cached_blocks":1348,"cache_hit":0,"new_prefill":4736,"score_linear":145452.0,"score_lmetric":129190},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":147211,"ongoing_decode_tokens":147211,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1117,"cache_hit":0,"new_prefill":4736,"score_linear":147211.0,"score_lmetric":9472}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348856.622260762,"t_first_token_unix":1779779721.636204,"t_done":348856.882026807,"t_done_unix":1779779721.8959696},{"request_id":"1394482:1:1394482:726","session_id":"1394482","input_length":4428,"t_proxy_recv":348856.577579245,"t_decision_unix":1779779721.5915208,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":4428,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":28665,"ongoing_decode_tokens":28665,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1511,"cache_hit":0,"new_prefill":4428,"score_linear":28665.0,"score_lmetric":4428},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1543,"cache_hit":0,"new_prefill":4428,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1983,"cache_hit":0,"new_prefill":4428,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":4736,"ongoing_decode_tokens":0,"pending_prefill_tokens":4736,"num_requests":1,"active_p_offloads":0,"cached_blocks":1927,"cache_hit":0,"new_prefill":4428,"score_linear":4736.0,"score_lmetric":9164},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":182626,"pending_prefill_tokens":761,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":4428,"score_linear":296027.0,"score_lmetric":15567},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":135124,"ongoing_decode_tokens":0,"pending_prefill_tokens":33748,"num_requests":2,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":4428,"score_linear":135124.0,"score_lmetric":76352},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":145452,"ongoing_decode_tokens":23641,"pending_prefill_tokens":59859,"num_requests":2,"active_p_offloads":0,"cached_blocks":1348,"cache_hit":0,"new_prefill":4428,"score_linear":145452.0,"score_lmetric":128574},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":147211,"ongoing_decode_tokens":147211,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1117,"cache_hit":0,"new_prefill":4428,"score_linear":147211.0,"score_lmetric":8856}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348856.852217399,"t_first_token_unix":1779779721.8661606,"t_done":348856.941556497,"t_done_unix":1779779721.9554994},{"request_id":"1279412:25:1366203:554","session_id":"1279412","input_length":98267,"t_proxy_recv":348794.679105919,"t_decision_unix":1779779659.6930473,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":0,"chosen_idx":7,"affinity_cache_hit":97792,"affinity_cache_ratio":0.9951662307794071,"affinity_num_requests":3,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":false,"cache_hit":12800,"estimated_new_tokens":85467,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":120133,"ongoing_decode_tokens":84245,"pending_prefill_tokens":48,"num_requests":3,"active_p_offloads":0,"cached_blocks":1381,"cache_hit":97792,"new_prefill":475,"score_linear":22341.0,"score_lmetric":1569},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1251,"cache_hit":0,"new_prefill":98267,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":164300,"ongoing_decode_tokens":0,"pending_prefill_tokens":92620,"num_requests":3,"active_p_offloads":0,"cached_blocks":1708,"cache_hit":0,"new_prefill":98267,"score_linear":164300.0,"score_lmetric":572661},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1792,"cache_hit":0,"new_prefill":98267,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":246162,"ongoing_decode_tokens":0,"pending_prefill_tokens":164754,"num_requests":2,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":98267,"score_linear":246162.0,"score_lmetric":526042},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":24323,"ongoing_decode_tokens":0,"pending_prefill_tokens":24323,"num_requests":1,"active_p_offloads":0,"cached_blocks":1252,"cache_hit":0,"new_prefill":98267,"score_linear":24323.0,"score_lmetric":122590},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":125018,"ongoing_decode_tokens":0,"pending_prefill_tokens":46170,"num_requests":2,"active_p_offloads":0,"cached_blocks":1249,"cache_hit":12800,"new_prefill":85467,"score_linear":112218.0,"score_lmetric":263274},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":12800,"new_prefill":85467,"score_linear":-12800.0,"score_lmetric":0}],"chosen_score_linear":-12800.0,"chosen_score_lmetric":0,"t_first_token":348826.323081184,"t_first_token_unix":1779779691.3370245,"t_done":348857.144482661,"t_done_unix":1779779722.158426},{"request_id":"1279412:26:1378269:628","session_id":"1279412","input_length":100340,"t_proxy_recv":348857.162292996,"t_decision_unix":1779779722.1762345,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":99840,"affinity_cache_ratio":0.9950169423958541,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":99840,"estimated_new_tokens":500,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1512,"cache_hit":97792,"new_prefill":2548,"score_linear":-97792.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1551,"cache_hit":0,"new_prefill":100340,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1983,"cache_hit":0,"new_prefill":100340,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1936,"cache_hit":0,"new_prefill":100340,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":182626,"pending_prefill_tokens":761,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":100340,"score_linear":296027.0,"score_lmetric":303303},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":135124,"ongoing_decode_tokens":0,"pending_prefill_tokens":33748,"num_requests":2,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":100340,"score_linear":135124.0,"score_lmetric":268176},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":145452,"ongoing_decode_tokens":23641,"pending_prefill_tokens":59859,"num_requests":2,"active_p_offloads":0,"cached_blocks":1348,"cache_hit":12800,"new_prefill":87540,"score_linear":132652.0,"score_lmetric":294798},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":48944,"ongoing_decode_tokens":48944,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1283,"cache_hit":99840,"new_prefill":500,"score_linear":-50896.0,"score_lmetric":500}],"chosen_score_linear":-50896.0,"chosen_score_lmetric":500,"t_first_token":348857.371557808,"t_first_token_unix":1779779722.385501,"t_done":348859.197867709,"t_done_unix":1779779724.2118113},{"request_id":"1394432:2:1395186:729","session_id":"1394432","input_length":6892,"t_proxy_recv":348858.909522397,"t_decision_unix":1779779723.9234638,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":4608,"affinity_cache_ratio":0.6686012768427162,"affinity_num_requests":0,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":4608,"estimated_new_tokens":2284,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":4863,"ongoing_decode_tokens":4863,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1512,"cache_hit":0,"new_prefill":6892,"score_linear":4863.0,"score_lmetric":6892},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":35450,"ongoing_decode_tokens":0,"pending_prefill_tokens":35450,"num_requests":1,"active_p_offloads":0,"cached_blocks":1551,"cache_hit":0,"new_prefill":6892,"score_linear":35450.0,"score_lmetric":42342},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1983,"cache_hit":0,"new_prefill":6892,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1936,"cache_hit":4608,"new_prefill":2284,"score_linear":-4608.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":246162,"pending_prefill_tokens":201,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":6892,"score_linear":296027.0,"score_lmetric":21279},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":135124,"ongoing_decode_tokens":0,"pending_prefill_tokens":33748,"num_requests":2,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":6892,"score_linear":135124.0,"score_lmetric":81280},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":145452,"ongoing_decode_tokens":23641,"pending_prefill_tokens":59859,"num_requests":2,"active_p_offloads":0,"cached_blocks":1348,"cache_hit":0,"new_prefill":6892,"score_linear":145452.0,"score_lmetric":133502},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":149284,"ongoing_decode_tokens":149284,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1283,"cache_hit":0,"new_prefill":6892,"score_linear":149284.0,"score_lmetric":13784}],"chosen_score_linear":-4608.0,"chosen_score_lmetric":0,"t_first_token":348859.108470291,"t_first_token_unix":1779779724.1224139,"t_done":348859.361722556,"t_done_unix":1779779724.375666},{"request_id":"1394167:2:1395165:728","session_id":"1394167","input_length":4863,"t_proxy_recv":348858.846115053,"t_decision_unix":1779779723.8600564,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":4608,"affinity_cache_ratio":0.9475632325724861,"affinity_num_requests":0,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":4608,"estimated_new_tokens":255,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1512,"cache_hit":4608,"new_prefill":255,"score_linear":-4608.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":35450,"ongoing_decode_tokens":0,"pending_prefill_tokens":35450,"num_requests":1,"active_p_offloads":0,"cached_blocks":1551,"cache_hit":0,"new_prefill":4863,"score_linear":35450.0,"score_lmetric":40313},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1983,"cache_hit":0,"new_prefill":4863,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1936,"cache_hit":0,"new_prefill":4863,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":246162,"pending_prefill_tokens":201,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":4863,"score_linear":296027.0,"score_lmetric":15192},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":135124,"ongoing_decode_tokens":0,"pending_prefill_tokens":33748,"num_requests":2,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":4863,"score_linear":135124.0,"score_lmetric":77222},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":145452,"ongoing_decode_tokens":23641,"pending_prefill_tokens":59859,"num_requests":2,"active_p_offloads":0,"cached_blocks":1348,"cache_hit":0,"new_prefill":4863,"score_linear":145452.0,"score_lmetric":129444},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":149284,"ongoing_decode_tokens":149284,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1283,"cache_hit":0,"new_prefill":4863,"score_linear":149284.0,"score_lmetric":9726}],"chosen_score_linear":-4608.0,"chosen_score_lmetric":0,"t_first_token":348858.88415283,"t_first_token_unix":1779779723.8980958,"t_done":348859.494013324,"t_done_unix":1779779724.507957},{"request_id":"1364090:4:1392998:718","session_id":"1364090","input_length":48944,"t_proxy_recv":348851.257039866,"t_decision_unix":1779779716.2709813,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":48128,"affinity_cache_ratio":0.9833278849297156,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":48128,"estimated_new_tokens":816,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":7440,"ongoing_decode_tokens":0,"pending_prefill_tokens":2320,"num_requests":1,"active_p_offloads":0,"cached_blocks":1498,"cache_hit":7680,"new_prefill":41264,"score_linear":-240.0,"score_lmetric":43584},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":92082,"ongoing_decode_tokens":0,"pending_prefill_tokens":73138,"num_requests":2,"active_p_offloads":0,"cached_blocks":1400,"cache_hit":0,"new_prefill":48944,"score_linear":92082.0,"score_lmetric":244164},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":84897,"ongoing_decode_tokens":84897,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1979,"cache_hit":0,"new_prefill":48944,"score_linear":84897.0,"score_lmetric":48944},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1927,"cache_hit":0,"new_prefill":48944,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":182626,"pending_prefill_tokens":761,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":7680,"new_prefill":41264,"score_linear":288347.0,"score_lmetric":126075},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":48944,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":85184,"ongoing_decode_tokens":85184,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1347,"cache_hit":7680,"new_prefill":41264,"score_linear":77504.0,"score_lmetric":82528},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":98267,"ongoing_decode_tokens":98267,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1117,"cache_hit":48128,"new_prefill":816,"score_linear":50139.0,"score_lmetric":816}],"chosen_score_linear":50139.0,"chosen_score_lmetric":816,"t_first_token":348851.640475523,"t_first_token_unix":1779779716.654419,"t_done":348860.310895597,"t_done_unix":1779779725.3248386},{"request_id":"1390240:3:1395387:730","session_id":"1390240","input_length":10890,"t_proxy_recv":348859.704348928,"t_decision_unix":1779779724.7182899,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":7168,"affinity_cache_ratio":0.65821854912764,"affinity_num_requests":0,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":7168,"estimated_new_tokens":3722,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1512,"cache_hit":7168,"new_prefill":3722,"score_linear":-7168.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":35450,"ongoing_decode_tokens":0,"pending_prefill_tokens":35450,"num_requests":1,"active_p_offloads":0,"cached_blocks":1551,"cache_hit":0,"new_prefill":10890,"score_linear":35450.0,"score_lmetric":46340},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1983,"cache_hit":0,"new_prefill":10890,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1940,"cache_hit":0,"new_prefill":10890,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":246162,"pending_prefill_tokens":201,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":10890,"score_linear":296027.0,"score_lmetric":33273},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":135124,"ongoing_decode_tokens":0,"pending_prefill_tokens":33748,"num_requests":2,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":10890,"score_linear":135124.0,"score_lmetric":89276},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":145452,"ongoing_decode_tokens":23641,"pending_prefill_tokens":59859,"num_requests":2,"active_p_offloads":0,"cached_blocks":1348,"cache_hit":0,"new_prefill":10890,"score_linear":145452.0,"score_lmetric":141498},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":149482,"ongoing_decode_tokens":149482,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1284,"cache_hit":0,"new_prefill":10890,"score_linear":149482.0,"score_lmetric":21780}],"chosen_score_linear":-7168.0,"chosen_score_lmetric":0,"t_first_token":348860.068554297,"t_first_token_unix":1779779725.0824978,"t_done":348860.649052476,"t_done_unix":1779779725.6629958},{"request_id":"1366128:17:1395596:731","session_id":"1366128","input_length":85276,"t_proxy_recv":348860.459105309,"t_decision_unix":1779779725.4730468,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":84992,"affinity_cache_ratio":0.9966696374126366,"affinity_num_requests":0,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":84992,"estimated_new_tokens":284,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":10890,"ongoing_decode_tokens":10890,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1512,"cache_hit":0,"new_prefill":85276,"score_linear":10890.0,"score_lmetric":85276},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":35450,"ongoing_decode_tokens":0,"pending_prefill_tokens":35450,"num_requests":1,"active_p_offloads":0,"cached_blocks":1551,"cache_hit":0,"new_prefill":85276,"score_linear":35450.0,"score_lmetric":120726},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1983,"cache_hit":84992,"new_prefill":284,"score_linear":-84992.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1940,"cache_hit":0,"new_prefill":85276,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":246162,"pending_prefill_tokens":201,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":85276,"score_linear":296027.0,"score_lmetric":256431},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":135124,"ongoing_decode_tokens":0,"pending_prefill_tokens":33748,"num_requests":2,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":85276,"score_linear":135124.0,"score_lmetric":238048},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":145452,"ongoing_decode_tokens":23641,"pending_prefill_tokens":59859,"num_requests":2,"active_p_offloads":0,"cached_blocks":1348,"cache_hit":0,"new_prefill":85276,"score_linear":145452.0,"score_lmetric":290270},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":100538,"ongoing_decode_tokens":100538,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1286,"cache_hit":0,"new_prefill":85276,"score_linear":100538.0,"score_lmetric":85276}],"chosen_score_linear":-84992.0,"chosen_score_lmetric":0,"t_first_token":348860.721404711,"t_first_token_unix":1779779725.735348,"t_done":348861.634695484,"t_done_unix":1779779726.648639},{"request_id":"1279412:27:1380696:646","session_id":"1279412","input_length":100538,"t_proxy_recv":348859.214182714,"t_decision_unix":1779779724.2281241,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":100352,"affinity_cache_ratio":0.9981499532515069,"affinity_num_requests":1,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":100352,"estimated_new_tokens":186,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":4863,"ongoing_decode_tokens":4863,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1512,"cache_hit":97792,"new_prefill":2746,"score_linear":-92929.0,"score_lmetric":2746},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":35450,"ongoing_decode_tokens":0,"pending_prefill_tokens":35450,"num_requests":1,"active_p_offloads":0,"cached_blocks":1551,"cache_hit":0,"new_prefill":100538,"score_linear":35450.0,"score_lmetric":135988},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1983,"cache_hit":0,"new_prefill":100538,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":6892,"ongoing_decode_tokens":6892,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1936,"cache_hit":0,"new_prefill":100538,"score_linear":6892.0,"score_lmetric":100538},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":246162,"pending_prefill_tokens":201,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":100538,"score_linear":296027.0,"score_lmetric":302217},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":135124,"ongoing_decode_tokens":0,"pending_prefill_tokens":33748,"num_requests":2,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":100538,"score_linear":135124.0,"score_lmetric":268572},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":145452,"ongoing_decode_tokens":23641,"pending_prefill_tokens":59859,"num_requests":2,"active_p_offloads":0,"cached_blocks":1348,"cache_hit":12800,"new_prefill":87738,"score_linear":132652.0,"score_lmetric":295194},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":48944,"ongoing_decode_tokens":48944,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1284,"cache_hit":100352,"new_prefill":186,"score_linear":-51408.0,"score_lmetric":186}],"chosen_score_linear":-51408.0,"chosen_score_lmetric":186,"t_first_token":348859.42844278,"t_first_token_unix":1779779724.4423862,"t_done":348861.65318774,"t_done_unix":1779779726.667131},{"request_id":"1394696:1:1394696:727","session_id":"1394696","input_length":35450,"t_proxy_recv":348857.350718061,"t_decision_unix":1779779722.3646595,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":35450,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1512,"cache_hit":0,"new_prefill":35450,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1551,"cache_hit":0,"new_prefill":35450,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1983,"cache_hit":0,"new_prefill":35450,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1936,"cache_hit":0,"new_prefill":35450,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":182626,"pending_prefill_tokens":761,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":35450,"score_linear":296027.0,"score_lmetric":108633},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":135124,"ongoing_decode_tokens":0,"pending_prefill_tokens":33748,"num_requests":2,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":35450,"score_linear":135124.0,"score_lmetric":138396},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":145452,"ongoing_decode_tokens":23641,"pending_prefill_tokens":59859,"num_requests":2,"active_p_offloads":0,"cached_blocks":1348,"cache_hit":0,"new_prefill":35450,"score_linear":145452.0,"score_lmetric":190618},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":149284,"ongoing_decode_tokens":48944,"pending_prefill_tokens":500,"num_requests":2,"active_p_offloads":0,"cached_blocks":1283,"cache_hit":0,"new_prefill":35450,"score_linear":149284.0,"score_lmetric":71900}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348862.534661378,"t_first_token_unix":1779779727.548605,"t_done":348862.959265964,"t_done_unix":1779779727.9732096},{"request_id":"1279412:28:1382526:657","session_id":"1279412","input_length":100766,"t_proxy_recv":348861.668614215,"t_decision_unix":1779779726.682556,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":100352,"affinity_cache_ratio":0.9958914713296152,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":100352,"estimated_new_tokens":414,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1519,"cache_hit":97792,"new_prefill":2974,"score_linear":-97792.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":35450,"ongoing_decode_tokens":0,"pending_prefill_tokens":35450,"num_requests":1,"active_p_offloads":0,"cached_blocks":1551,"cache_hit":0,"new_prefill":100766,"score_linear":35450.0,"score_lmetric":136216},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":21241,"ongoing_decode_tokens":0,"pending_prefill_tokens":9465,"num_requests":1,"active_p_offloads":0,"cached_blocks":1983,"cache_hit":0,"new_prefill":100766,"score_linear":21241.0,"score_lmetric":110231},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1940,"cache_hit":0,"new_prefill":100766,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":246162,"pending_prefill_tokens":201,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":100766,"score_linear":296027.0,"score_lmetric":302901},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":135124,"ongoing_decode_tokens":0,"pending_prefill_tokens":33748,"num_requests":2,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":100766,"score_linear":135124.0,"score_lmetric":269028},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":145452,"ongoing_decode_tokens":23641,"pending_prefill_tokens":59859,"num_requests":2,"active_p_offloads":0,"cached_blocks":1348,"cache_hit":12800,"new_prefill":87966,"score_linear":132652.0,"score_lmetric":295650},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1286,"cache_hit":100352,"new_prefill":414,"score_linear":-100352.0,"score_lmetric":0}],"chosen_score_linear":-100352.0,"chosen_score_lmetric":0,"t_first_token":348861.81998376,"t_first_token_unix":1779779726.833928,"t_done":348863.902093764,"t_done_unix":1779779728.9160423},{"request_id":"1279412:29:1385688:685","session_id":"1279412","input_length":101200,"t_proxy_recv":348863.917344829,"t_decision_unix":1779779728.9312863,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":100864,"affinity_cache_ratio":0.9966798418972332,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":100864,"estimated_new_tokens":336,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1519,"cache_hit":97792,"new_prefill":3408,"score_linear":-97792.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1620,"cache_hit":0,"new_prefill":101200,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":21241,"ongoing_decode_tokens":21241,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1983,"cache_hit":0,"new_prefill":101200,"score_linear":21241.0,"score_lmetric":101200},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1940,"cache_hit":0,"new_prefill":101200,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":246162,"pending_prefill_tokens":201,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":101200,"score_linear":296027.0,"score_lmetric":304203},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":135124,"ongoing_decode_tokens":0,"pending_prefill_tokens":33748,"num_requests":2,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":101200,"score_linear":135124.0,"score_lmetric":269896},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":145452,"ongoing_decode_tokens":23641,"pending_prefill_tokens":59859,"num_requests":2,"active_p_offloads":0,"cached_blocks":1348,"cache_hit":12800,"new_prefill":88400,"score_linear":132652.0,"score_lmetric":296518},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1287,"cache_hit":100864,"new_prefill":336,"score_linear":-100864.0,"score_lmetric":0}],"chosen_score_linear":-100864.0,"chosen_score_lmetric":0,"t_first_token":348864.225205734,"t_first_token_unix":1779779729.2391496,"t_done":348864.932482516,"t_done_unix":1779779729.9464269},{"request_id":"1395933:1:1395933:732","session_id":"1395933","input_length":21241,"t_proxy_recv":348861.64447849,"t_decision_unix":1779779726.65842,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":false,"cache_hit":11776,"estimated_new_tokens":9465,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1519,"cache_hit":0,"new_prefill":21241,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":35450,"ongoing_decode_tokens":0,"pending_prefill_tokens":35450,"num_requests":1,"active_p_offloads":0,"cached_blocks":1551,"cache_hit":0,"new_prefill":21241,"score_linear":35450.0,"score_lmetric":56691},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1983,"cache_hit":11776,"new_prefill":9465,"score_linear":-11776.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1940,"cache_hit":0,"new_prefill":21241,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":246162,"pending_prefill_tokens":201,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":21241,"score_linear":296027.0,"score_lmetric":64326},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":135124,"ongoing_decode_tokens":0,"pending_prefill_tokens":33748,"num_requests":2,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":21241,"score_linear":135124.0,"score_lmetric":109978},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":145452,"ongoing_decode_tokens":23641,"pending_prefill_tokens":59859,"num_requests":2,"active_p_offloads":0,"cached_blocks":1348,"cache_hit":11776,"new_prefill":9465,"score_linear":133676.0,"score_lmetric":138648},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":100538,"ongoing_decode_tokens":100538,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1286,"cache_hit":0,"new_prefill":21241,"score_linear":100538.0,"score_lmetric":21241}],"chosen_score_linear":-11776.0,"chosen_score_lmetric":0,"t_first_token":348863.017088136,"t_first_token_unix":1779779728.0310316,"t_done":348865.934068273,"t_done_unix":1779779730.9480119},{"request_id":"1279412:30:1387344:694","session_id":"1279412","input_length":101396,"t_proxy_recv":348864.956247199,"t_decision_unix":1779779729.9701889,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":100864,"affinity_cache_ratio":0.9947532447039331,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":100864,"estimated_new_tokens":532,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1519,"cache_hit":97792,"new_prefill":3604,"score_linear":-97792.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1620,"cache_hit":0,"new_prefill":101396,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":38288,"ongoing_decode_tokens":21241,"pending_prefill_tokens":663,"num_requests":2,"active_p_offloads":0,"cached_blocks":1983,"cache_hit":0,"new_prefill":101396,"score_linear":38288.0,"score_lmetric":204118},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1940,"cache_hit":0,"new_prefill":101396,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":246162,"pending_prefill_tokens":201,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":101396,"score_linear":296027.0,"score_lmetric":304791},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":135124,"ongoing_decode_tokens":0,"pending_prefill_tokens":33748,"num_requests":2,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":101396,"score_linear":135124.0,"score_lmetric":270288},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":145452,"ongoing_decode_tokens":23641,"pending_prefill_tokens":59859,"num_requests":2,"active_p_offloads":0,"cached_blocks":1348,"cache_hit":12800,"new_prefill":88596,"score_linear":132652.0,"score_lmetric":296910},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1287,"cache_hit":100864,"new_prefill":532,"score_linear":-100864.0,"score_lmetric":0}],"chosen_score_linear":-100864.0,"chosen_score_lmetric":0,"t_first_token":348865.181847049,"t_first_token_unix":1779779730.1957903,"t_done":348865.976913157,"t_done_unix":1779779730.9908574},{"request_id":"1305906:3:1396717:733","session_id":"1305906","input_length":17047,"t_proxy_recv":348864.238569304,"t_decision_unix":1779779729.2525103,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":16384,"affinity_cache_ratio":0.9611075262509532,"affinity_num_requests":1,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":16384,"estimated_new_tokens":663,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1519,"cache_hit":0,"new_prefill":17047,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1620,"cache_hit":0,"new_prefill":17047,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":21241,"ongoing_decode_tokens":21241,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1983,"cache_hit":16384,"new_prefill":663,"score_linear":4857.0,"score_lmetric":663},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1940,"cache_hit":0,"new_prefill":17047,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":246162,"pending_prefill_tokens":201,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":17047,"score_linear":296027.0,"score_lmetric":51744},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":135124,"ongoing_decode_tokens":0,"pending_prefill_tokens":33748,"num_requests":2,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":17047,"score_linear":135124.0,"score_lmetric":101590},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":145452,"ongoing_decode_tokens":23641,"pending_prefill_tokens":59859,"num_requests":2,"active_p_offloads":0,"cached_blocks":1348,"cache_hit":0,"new_prefill":17047,"score_linear":145452.0,"score_lmetric":153812},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":101200,"ongoing_decode_tokens":101200,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1287,"cache_hit":0,"new_prefill":17047,"score_linear":101200.0,"score_lmetric":17047}],"chosen_score_linear":4857.0,"chosen_score_lmetric":663,"t_first_token":348865.889243475,"t_first_token_unix":1779779730.9031873,"t_done":348866.885770452,"t_done_unix":1779779731.899714},{"request_id":"1366128:18:1397382:735","session_id":"1366128","input_length":85504,"t_proxy_recv":348866.512733794,"t_decision_unix":1779779731.5266752,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":84992,"affinity_cache_ratio":0.9940119760479041,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":84992,"estimated_new_tokens":512,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1519,"cache_hit":0,"new_prefill":85504,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1620,"cache_hit":0,"new_prefill":85504,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":17047,"ongoing_decode_tokens":17047,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2001,"cache_hit":84992,"new_prefill":512,"score_linear":-67945.0,"score_lmetric":512},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":13134,"ongoing_decode_tokens":0,"pending_prefill_tokens":13134,"num_requests":1,"active_p_offloads":0,"cached_blocks":1940,"cache_hit":0,"new_prefill":85504,"score_linear":13134.0,"score_lmetric":98638},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":246162,"pending_prefill_tokens":201,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":85504,"score_linear":296027.0,"score_lmetric":257115},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":135124,"ongoing_decode_tokens":0,"pending_prefill_tokens":33748,"num_requests":2,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":85504,"score_linear":135124.0,"score_lmetric":238504},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":145452,"ongoing_decode_tokens":23641,"pending_prefill_tokens":59859,"num_requests":2,"active_p_offloads":0,"cached_blocks":1348,"cache_hit":0,"new_prefill":85504,"score_linear":145452.0,"score_lmetric":290726},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":101993,"ongoing_decode_tokens":101993,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1288,"cache_hit":0,"new_prefill":85504,"score_linear":101993.0,"score_lmetric":85504}],"chosen_score_linear":-67945.0,"chosen_score_lmetric":512,"t_first_token":348866.70310084,"t_first_token_unix":1779779731.7170446,"t_done":348867.533928351,"t_done_unix":1779779732.5478716},{"request_id":"1279412:31:1391216:711","session_id":"1279412","input_length":101993,"t_proxy_recv":348865.994543831,"t_decision_unix":1779779731.008485,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":101376,"affinity_cache_ratio":0.993950565234869,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":101376,"estimated_new_tokens":617,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1519,"cache_hit":97792,"new_prefill":4201,"score_linear":-97792.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1620,"cache_hit":0,"new_prefill":101993,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":17047,"ongoing_decode_tokens":17047,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2001,"cache_hit":0,"new_prefill":101993,"score_linear":17047.0,"score_lmetric":101993},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":13134,"ongoing_decode_tokens":0,"pending_prefill_tokens":13134,"num_requests":1,"active_p_offloads":0,"cached_blocks":1940,"cache_hit":0,"new_prefill":101993,"score_linear":13134.0,"score_lmetric":115127},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":246162,"pending_prefill_tokens":201,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":101993,"score_linear":296027.0,"score_lmetric":306582},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":135124,"ongoing_decode_tokens":0,"pending_prefill_tokens":33748,"num_requests":2,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":101993,"score_linear":135124.0,"score_lmetric":271482},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":145452,"ongoing_decode_tokens":23641,"pending_prefill_tokens":59859,"num_requests":2,"active_p_offloads":0,"cached_blocks":1348,"cache_hit":12800,"new_prefill":89193,"score_linear":132652.0,"score_lmetric":298104},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1288,"cache_hit":101376,"new_prefill":617,"score_linear":-101376.0,"score_lmetric":0}],"chosen_score_linear":-101376.0,"chosen_score_lmetric":0,"t_first_token":348866.49797021,"t_first_token_unix":1779779731.5119143,"t_done":348869.267669738,"t_done_unix":1779779734.2816133},{"request_id":"1398213:1:1398213:742","session_id":"1398213","input_length":3891,"t_proxy_recv":348869.536538448,"t_decision_unix":1779779734.5504797,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.625,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":3891,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":34437,"ongoing_decode_tokens":0,"pending_prefill_tokens":5765,"num_requests":2,"active_p_offloads":0,"cached_blocks":1519,"cache_hit":0,"new_prefill":3891,"score_linear":34437.0,"score_lmetric":19312},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":93040,"ongoing_decode_tokens":0,"pending_prefill_tokens":92528,"num_requests":1,"active_p_offloads":0,"cached_blocks":1620,"cache_hit":0,"new_prefill":3891,"score_linear":93040.0,"score_lmetric":96419},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":27286,"ongoing_decode_tokens":0,"pending_prefill_tokens":17046,"num_requests":2,"active_p_offloads":0,"cached_blocks":2003,"cache_hit":0,"new_prefill":3891,"score_linear":27286.0,"score_lmetric":41874},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":13134,"ongoing_decode_tokens":13134,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1940,"cache_hit":0,"new_prefill":3891,"score_linear":13134.0,"score_lmetric":3891},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":246162,"pending_prefill_tokens":201,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":3891,"score_linear":296027.0,"score_lmetric":12276},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":135124,"ongoing_decode_tokens":0,"pending_prefill_tokens":33748,"num_requests":2,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":3891,"score_linear":135124.0,"score_lmetric":75278},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":145452,"ongoing_decode_tokens":23641,"pending_prefill_tokens":59859,"num_requests":2,"active_p_offloads":0,"cached_blocks":1348,"cache_hit":0,"new_prefill":3891,"score_linear":145452.0,"score_lmetric":127500},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1289,"cache_hit":0,"new_prefill":3891,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348869.778123303,"t_first_token_unix":1779779734.7920668,"t_done":348870.104461436,"t_done_unix":1779779735.1184046},{"request_id":"1398145:1:1398145:739","session_id":"1398145","input_length":3213,"t_proxy_recv":348869.395850391,"t_decision_unix":1779779734.4097917,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":3213,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1519,"cache_hit":0,"new_prefill":3213,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":93040,"ongoing_decode_tokens":0,"pending_prefill_tokens":92528,"num_requests":1,"active_p_offloads":0,"cached_blocks":1620,"cache_hit":0,"new_prefill":3213,"score_linear":93040.0,"score_lmetric":95741},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":16550,"ongoing_decode_tokens":0,"pending_prefill_tokens":16550,"num_requests":1,"active_p_offloads":0,"cached_blocks":2003,"cache_hit":0,"new_prefill":3213,"score_linear":16550.0,"score_lmetric":19763},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":13134,"ongoing_decode_tokens":13134,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1940,"cache_hit":0,"new_prefill":3213,"score_linear":13134.0,"score_lmetric":3213},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":246162,"pending_prefill_tokens":201,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":3213,"score_linear":296027.0,"score_lmetric":10242},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":135124,"ongoing_decode_tokens":0,"pending_prefill_tokens":33748,"num_requests":2,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":3213,"score_linear":135124.0,"score_lmetric":73922},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":145452,"ongoing_decode_tokens":23641,"pending_prefill_tokens":59859,"num_requests":2,"active_p_offloads":0,"cached_blocks":1348,"cache_hit":0,"new_prefill":3213,"score_linear":145452.0,"score_lmetric":126144},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1289,"cache_hit":0,"new_prefill":3213,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348869.597870567,"t_first_token_unix":1779779734.6118143,"t_done":348870.334654525,"t_done_unix":1779779735.348598},{"request_id":"1362265:5:1398180:741","session_id":"1362265","input_length":10736,"t_proxy_recv":348869.481588076,"t_decision_unix":1779779734.4955292,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":10240,"affinity_cache_ratio":0.9538002980625931,"affinity_num_requests":1,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":10240,"estimated_new_tokens":496,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":3213,"ongoing_decode_tokens":0,"pending_prefill_tokens":3213,"num_requests":1,"active_p_offloads":0,"cached_blocks":1519,"cache_hit":0,"new_prefill":10736,"score_linear":3213.0,"score_lmetric":13949},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":93040,"ongoing_decode_tokens":0,"pending_prefill_tokens":92528,"num_requests":1,"active_p_offloads":0,"cached_blocks":1620,"cache_hit":0,"new_prefill":10736,"score_linear":93040.0,"score_lmetric":103264},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":16550,"ongoing_decode_tokens":0,"pending_prefill_tokens":16550,"num_requests":1,"active_p_offloads":0,"cached_blocks":2003,"cache_hit":10240,"new_prefill":496,"score_linear":6310.0,"score_lmetric":17046},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":13134,"ongoing_decode_tokens":13134,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1940,"cache_hit":0,"new_prefill":10736,"score_linear":13134.0,"score_lmetric":10736},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":246162,"pending_prefill_tokens":201,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":10736,"score_linear":296027.0,"score_lmetric":32811},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":135124,"ongoing_decode_tokens":0,"pending_prefill_tokens":33748,"num_requests":2,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":10736,"score_linear":135124.0,"score_lmetric":88968},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":145452,"ongoing_decode_tokens":23641,"pending_prefill_tokens":59859,"num_requests":2,"active_p_offloads":0,"cached_blocks":1348,"cache_hit":0,"new_prefill":10736,"score_linear":145452.0,"score_lmetric":141190},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1289,"cache_hit":0,"new_prefill":10736,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":6310.0,"chosen_score_lmetric":17046,"t_first_token":348870.712954931,"t_first_token_unix":1779779735.7268984,"t_done":348870.944260611,"t_done_unix":1779779735.9582038},{"request_id":"1397840:1:1397840:737","session_id":"1397840","input_length":16550,"t_proxy_recv":348868.326849144,"t_decision_unix":1779779733.3407905,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":2,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":16550,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1519,"cache_hit":0,"new_prefill":16550,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":93040,"ongoing_decode_tokens":0,"pending_prefill_tokens":92528,"num_requests":1,"active_p_offloads":0,"cached_blocks":1620,"cache_hit":0,"new_prefill":16550,"score_linear":93040.0,"score_lmetric":109078},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2003,"cache_hit":0,"new_prefill":16550,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":13134,"ongoing_decode_tokens":13134,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1940,"cache_hit":0,"new_prefill":16550,"score_linear":13134.0,"score_lmetric":16550},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":246162,"pending_prefill_tokens":201,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":16550,"score_linear":296027.0,"score_lmetric":50253},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":135124,"ongoing_decode_tokens":0,"pending_prefill_tokens":33748,"num_requests":2,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":16550,"score_linear":135124.0,"score_lmetric":100596},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":145452,"ongoing_decode_tokens":23641,"pending_prefill_tokens":59859,"num_requests":2,"active_p_offloads":0,"cached_blocks":1348,"cache_hit":0,"new_prefill":16550,"score_linear":145452.0,"score_lmetric":152818},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":101993,"ongoing_decode_tokens":101993,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1288,"cache_hit":0,"new_prefill":16550,"score_linear":101993.0,"score_lmetric":16550}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348869.884191706,"t_first_token_unix":1779779734.8981352,"t_done":348870.997169209,"t_done_unix":1779779736.0111127},{"request_id":"1398594:1:1398594:744","session_id":"1398594","input_length":7877,"t_proxy_recv":348870.764072244,"t_decision_unix":1779779735.7780135,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.625,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":7877,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":31224,"ongoing_decode_tokens":31224,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1525,"cache_hit":0,"new_prefill":7877,"score_linear":31224.0,"score_lmetric":7877},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":93040,"ongoing_decode_tokens":0,"pending_prefill_tokens":92528,"num_requests":1,"active_p_offloads":0,"cached_blocks":1620,"cache_hit":0,"new_prefill":7877,"score_linear":93040.0,"score_lmetric":100405},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":27286,"ongoing_decode_tokens":27286,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":2003,"cache_hit":0,"new_prefill":7877,"score_linear":27286.0,"score_lmetric":15754},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76392,"ongoing_decode_tokens":13134,"pending_prefill_tokens":12058,"num_requests":2,"active_p_offloads":0,"cached_blocks":1940,"cache_hit":0,"new_prefill":7877,"score_linear":76392.0,"score_lmetric":39870},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":246162,"pending_prefill_tokens":201,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":7877,"score_linear":296027.0,"score_lmetric":24234},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":135124,"ongoing_decode_tokens":0,"pending_prefill_tokens":33748,"num_requests":2,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":7877,"score_linear":135124.0,"score_lmetric":83250},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":145452,"ongoing_decode_tokens":23641,"pending_prefill_tokens":59859,"num_requests":2,"active_p_offloads":0,"cached_blocks":1348,"cache_hit":0,"new_prefill":7877,"score_linear":145452.0,"score_lmetric":135472},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1296,"cache_hit":0,"new_prefill":7877,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348871.307656965,"t_first_token_unix":1779779736.3216007,"t_done":348871.424709739,"t_done_unix":1779779736.4386532},{"request_id":"1355484:14:1398178:740","session_id":"1355484","input_length":31224,"t_proxy_recv":348869.485667346,"t_decision_unix":1779779734.499609,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":28672,"affinity_cache_ratio":0.9182679989751473,"affinity_num_requests":1,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":28672,"estimated_new_tokens":2552,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":3213,"ongoing_decode_tokens":0,"pending_prefill_tokens":3213,"num_requests":1,"active_p_offloads":0,"cached_blocks":1519,"cache_hit":28672,"new_prefill":2552,"score_linear":-25459.0,"score_lmetric":5765},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":93040,"ongoing_decode_tokens":0,"pending_prefill_tokens":92528,"num_requests":1,"active_p_offloads":0,"cached_blocks":1620,"cache_hit":0,"new_prefill":31224,"score_linear":93040.0,"score_lmetric":123752},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":27286,"ongoing_decode_tokens":0,"pending_prefill_tokens":17046,"num_requests":2,"active_p_offloads":0,"cached_blocks":2003,"cache_hit":0,"new_prefill":31224,"score_linear":27286.0,"score_lmetric":96540},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":13134,"ongoing_decode_tokens":13134,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1940,"cache_hit":0,"new_prefill":31224,"score_linear":13134.0,"score_lmetric":31224},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":246162,"pending_prefill_tokens":201,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":31224,"score_linear":296027.0,"score_lmetric":94275},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":135124,"ongoing_decode_tokens":0,"pending_prefill_tokens":33748,"num_requests":2,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":31224,"score_linear":135124.0,"score_lmetric":129944},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":145452,"ongoing_decode_tokens":23641,"pending_prefill_tokens":59859,"num_requests":2,"active_p_offloads":0,"cached_blocks":1348,"cache_hit":12800,"new_prefill":18424,"score_linear":132652.0,"score_lmetric":156566},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1289,"cache_hit":12800,"new_prefill":18424,"score_linear":-12800.0,"score_lmetric":0}],"chosen_score_linear":-25459.0,"chosen_score_lmetric":5765,"t_first_token":348870.202138283,"t_first_token_unix":1779779735.216082,"t_done":348871.635739581,"t_done_unix":1779779736.649688},{"request_id":"1366128:19:1398824:745","session_id":"1366128","input_length":85663,"t_proxy_recv":348871.66114094,"t_decision_unix":1779779736.6750824,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":85504,"affinity_cache_ratio":0.9981438894271739,"affinity_num_requests":0,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":85504,"estimated_new_tokens":159,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1530,"cache_hit":0,"new_prefill":85663,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":93040,"ongoing_decode_tokens":0,"pending_prefill_tokens":92528,"num_requests":1,"active_p_offloads":0,"cached_blocks":1620,"cache_hit":0,"new_prefill":85663,"score_linear":93040.0,"score_lmetric":178191},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2036,"cache_hit":85504,"new_prefill":159,"score_linear":-85504.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76392,"ongoing_decode_tokens":13134,"pending_prefill_tokens":12058,"num_requests":2,"active_p_offloads":0,"cached_blocks":1940,"cache_hit":0,"new_prefill":85663,"score_linear":76392.0,"score_lmetric":195442},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":246162,"pending_prefill_tokens":201,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":85663,"score_linear":296027.0,"score_lmetric":257592},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":135124,"ongoing_decode_tokens":0,"pending_prefill_tokens":33748,"num_requests":2,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":85663,"score_linear":135124.0,"score_lmetric":238822},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":145452,"ongoing_decode_tokens":23641,"pending_prefill_tokens":59859,"num_requests":2,"active_p_offloads":0,"cached_blocks":1348,"cache_hit":0,"new_prefill":85663,"score_linear":145452.0,"score_lmetric":291044},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1311,"cache_hit":0,"new_prefill":85663,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-85504.0,"chosen_score_lmetric":0,"t_first_token":348871.842132543,"t_first_token_unix":1779779736.8560765,"t_done":348872.030159409,"t_done_unix":1779779737.0441031},{"request_id":"1362265:6:1399937:747","session_id":"1362265","input_length":10816,"t_proxy_recv":348875.543912124,"t_decision_unix":1779779740.5578535,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":10752,"affinity_cache_ratio":0.9940828402366864,"affinity_num_requests":0,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":10752,"estimated_new_tokens":64,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1530,"cache_hit":0,"new_prefill":10816,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":93040,"ongoing_decode_tokens":0,"pending_prefill_tokens":92528,"num_requests":1,"active_p_offloads":0,"cached_blocks":1620,"cache_hit":0,"new_prefill":10816,"score_linear":93040.0,"score_lmetric":103344},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2036,"cache_hit":10752,"new_prefill":64,"score_linear":-10752.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76392,"ongoing_decode_tokens":13134,"pending_prefill_tokens":12058,"num_requests":2,"active_p_offloads":0,"cached_blocks":1940,"cache_hit":0,"new_prefill":10816,"score_linear":76392.0,"score_lmetric":45748},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":246162,"pending_prefill_tokens":201,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":10816,"score_linear":296027.0,"score_lmetric":33051},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":135124,"ongoing_decode_tokens":0,"pending_prefill_tokens":33748,"num_requests":2,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":10816,"score_linear":135124.0,"score_lmetric":89128},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":145452,"ongoing_decode_tokens":23641,"pending_prefill_tokens":59859,"num_requests":2,"active_p_offloads":0,"cached_blocks":1348,"cache_hit":0,"new_prefill":10816,"score_linear":145452.0,"score_lmetric":141350},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1311,"cache_hit":0,"new_prefill":10816,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-10752.0,"chosen_score_lmetric":0,"t_first_token":348875.584047758,"t_first_token_unix":1779779740.597991,"t_done":348875.824030142,"t_done_unix":1779779740.8379738},{"request_id":"1399948:1:1399948:748","session_id":"1399948","input_length":4991,"t_proxy_recv":348875.574250203,"t_decision_unix":1779779740.5881913,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":4991,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1530,"cache_hit":0,"new_prefill":4991,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":93040,"ongoing_decode_tokens":0,"pending_prefill_tokens":92528,"num_requests":1,"active_p_offloads":0,"cached_blocks":1620,"cache_hit":0,"new_prefill":4991,"score_linear":93040.0,"score_lmetric":97519},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":10816,"ongoing_decode_tokens":0,"pending_prefill_tokens":64,"num_requests":1,"active_p_offloads":0,"cached_blocks":2036,"cache_hit":0,"new_prefill":4991,"score_linear":10816.0,"score_lmetric":5055},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76392,"ongoing_decode_tokens":13134,"pending_prefill_tokens":12058,"num_requests":2,"active_p_offloads":0,"cached_blocks":1940,"cache_hit":0,"new_prefill":4991,"score_linear":76392.0,"score_lmetric":34098},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":246162,"pending_prefill_tokens":201,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":4991,"score_linear":296027.0,"score_lmetric":15576},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":135124,"ongoing_decode_tokens":0,"pending_prefill_tokens":33748,"num_requests":2,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":4991,"score_linear":135124.0,"score_lmetric":77478},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":145452,"ongoing_decode_tokens":23641,"pending_prefill_tokens":59859,"num_requests":2,"active_p_offloads":0,"cached_blocks":1348,"cache_hit":0,"new_prefill":4991,"score_linear":145452.0,"score_lmetric":129700},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1311,"cache_hit":0,"new_prefill":4991,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348875.88880089,"t_first_token_unix":1779779740.9027455,"t_done":348876.557533767,"t_done_unix":1779779741.5714767},{"request_id":"1355484:15:1400281:750","session_id":"1355484","input_length":31469,"t_proxy_recv":348876.581162258,"t_decision_unix":1779779741.5951035,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":31232,"affinity_cache_ratio":0.9924687787981823,"affinity_num_requests":0,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":31232,"estimated_new_tokens":237,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1530,"cache_hit":31232,"new_prefill":237,"score_linear":-31232.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":93040,"ongoing_decode_tokens":0,"pending_prefill_tokens":92528,"num_requests":1,"active_p_offloads":0,"cached_blocks":1620,"cache_hit":0,"new_prefill":31469,"score_linear":93040.0,"score_lmetric":123997},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":26440,"ongoing_decode_tokens":0,"pending_prefill_tokens":10056,"num_requests":1,"active_p_offloads":0,"cached_blocks":2036,"cache_hit":0,"new_prefill":31469,"score_linear":26440.0,"score_lmetric":41525},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76392,"ongoing_decode_tokens":13134,"pending_prefill_tokens":12058,"num_requests":2,"active_p_offloads":0,"cached_blocks":1940,"cache_hit":0,"new_prefill":31469,"score_linear":76392.0,"score_lmetric":87054},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":246162,"pending_prefill_tokens":201,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":31469,"score_linear":296027.0,"score_lmetric":95010},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":135124,"ongoing_decode_tokens":0,"pending_prefill_tokens":33748,"num_requests":2,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":31469,"score_linear":135124.0,"score_lmetric":130434},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":145452,"ongoing_decode_tokens":23641,"pending_prefill_tokens":59859,"num_requests":2,"active_p_offloads":0,"cached_blocks":1348,"cache_hit":12800,"new_prefill":18669,"score_linear":132652.0,"score_lmetric":157056},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":12800,"new_prefill":18669,"score_linear":-12800.0,"score_lmetric":0}],"chosen_score_linear":-31232.0,"chosen_score_lmetric":0,"t_first_token":348876.646804925,"t_first_token_unix":1779779741.6607482,"t_done":348877.15428912,"t_done_unix":1779779742.1682324},{"request_id":"1400379:1:1400379:751","session_id":"1400379","input_length":3707,"t_proxy_recv":348876.891229225,"t_decision_unix":1779779741.9051707,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":7,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.5,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":3707,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":31469,"ongoing_decode_tokens":31469,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1530,"cache_hit":0,"new_prefill":3707,"score_linear":31469.0,"score_lmetric":3707},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":93040,"ongoing_decode_tokens":0,"pending_prefill_tokens":92528,"num_requests":1,"active_p_offloads":0,"cached_blocks":1620,"cache_hit":0,"new_prefill":3707,"score_linear":93040.0,"score_lmetric":96235},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":26440,"ongoing_decode_tokens":0,"pending_prefill_tokens":10056,"num_requests":1,"active_p_offloads":0,"cached_blocks":2036,"cache_hit":0,"new_prefill":3707,"score_linear":26440.0,"score_lmetric":13763},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76392,"ongoing_decode_tokens":13134,"pending_prefill_tokens":12058,"num_requests":2,"active_p_offloads":0,"cached_blocks":1940,"cache_hit":0,"new_prefill":3707,"score_linear":76392.0,"score_lmetric":31530},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":246162,"pending_prefill_tokens":201,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":3707,"score_linear":296027.0,"score_lmetric":11724},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":135124,"ongoing_decode_tokens":0,"pending_prefill_tokens":33748,"num_requests":2,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":3707,"score_linear":135124.0,"score_lmetric":74910},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":145452,"ongoing_decode_tokens":23641,"pending_prefill_tokens":59859,"num_requests":2,"active_p_offloads":0,"cached_blocks":1348,"cache_hit":0,"new_prefill":3707,"score_linear":145452.0,"score_lmetric":127132},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":3707,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348877.124007131,"t_first_token_unix":1779779742.137951,"t_done":348877.410528465,"t_done_unix":1779779742.4244716},{"request_id":"1373431:1:1373431:606","session_id":"1373431","input_length":126340,"t_proxy_recv":348782.770826619,"t_decision_unix":1779779647.784768,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":4,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.0,"fallback_score":-126340,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":126340,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":304659,"ongoing_decode_tokens":240128,"pending_prefill_tokens":5651,"num_requests":4,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":0,"new_prefill":126340,"score_linear":304659.0,"score_lmetric":527964},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":132609,"ongoing_decode_tokens":132609,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":126340,"score_linear":132609.0,"score_lmetric":252680},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1708,"cache_hit":0,"new_prefill":126340,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1750,"cache_hit":1024,"new_prefill":125316,"score_linear":-1024.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":126340,"score_linear":0.0,"score_lmetric":-126340},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1214,"cache_hit":0,"new_prefill":126340,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":60004,"ongoing_decode_tokens":60004,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1246,"cache_hit":0,"new_prefill":126340,"score_linear":60004.0,"score_lmetric":126340},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":106951,"ongoing_decode_tokens":106951,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1047,"cache_hit":0,"new_prefill":126340,"score_linear":106951.0,"score_lmetric":252680}],"chosen_score_linear":0.0,"chosen_score_lmetric":-126340,"t_first_token":348834.523213694,"t_first_token_unix":1779779699.5371602,"t_done":348878.205251598,"t_done_unix":1779779743.2191951},{"request_id":"1398213:2:1400590:753","session_id":"1398213","input_length":11414,"t_proxy_recv":348877.678432891,"t_decision_unix":1779779742.6923742,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":3584,"affinity_cache_ratio":0.3140003504468197,"affinity_num_requests":0,"avg_num_requests":1.5,"fallback_score":0,"tie_break_used":false,"cache_hit":3584,"estimated_new_tokens":7830,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1530,"cache_hit":0,"new_prefill":11414,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":93040,"ongoing_decode_tokens":0,"pending_prefill_tokens":92528,"num_requests":1,"active_p_offloads":0,"cached_blocks":1620,"cache_hit":0,"new_prefill":11414,"score_linear":93040.0,"score_lmetric":103942},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":124097,"ongoing_decode_tokens":0,"pending_prefill_tokens":22209,"num_requests":2,"active_p_offloads":0,"cached_blocks":2036,"cache_hit":0,"new_prefill":11414,"score_linear":124097.0,"score_lmetric":67246},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76392,"ongoing_decode_tokens":13134,"pending_prefill_tokens":12058,"num_requests":2,"active_p_offloads":0,"cached_blocks":1940,"cache_hit":0,"new_prefill":11414,"score_linear":76392.0,"score_lmetric":46944},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":246162,"pending_prefill_tokens":201,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":11414,"score_linear":296027.0,"score_lmetric":34845},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":135124,"ongoing_decode_tokens":0,"pending_prefill_tokens":33748,"num_requests":2,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":11414,"score_linear":135124.0,"score_lmetric":90324},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":145452,"ongoing_decode_tokens":23641,"pending_prefill_tokens":59859,"num_requests":2,"active_p_offloads":0,"cached_blocks":1348,"cache_hit":0,"new_prefill":11414,"score_linear":145452.0,"score_lmetric":142546},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1327,"cache_hit":3584,"new_prefill":7830,"score_linear":-3584.0,"score_lmetric":0}],"chosen_score_linear":-3584.0,"chosen_score_lmetric":0,"t_first_token":348878.348697956,"t_first_token_unix":1779779743.3626409,"t_done":348878.749300951,"t_done_unix":1779779743.7632437},{"request_id":"1370129:2:1373467:607","session_id":"1370129","input_length":56286,"t_proxy_recv":348782.862668752,"t_decision_unix":1779779647.87661,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":18432,"affinity_cache_ratio":0.3274704189318836,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":false,"cache_hit":18432,"estimated_new_tokens":37854,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":304659,"ongoing_decode_tokens":240128,"pending_prefill_tokens":5651,"num_requests":4,"active_p_offloads":0,"cached_blocks":1144,"cache_hit":7680,"new_prefill":48606,"score_linear":296979.0,"score_lmetric":217028},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":132609,"ongoing_decode_tokens":132609,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1004,"cache_hit":0,"new_prefill":56286,"score_linear":132609.0,"score_lmetric":112572},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1708,"cache_hit":0,"new_prefill":56286,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1750,"cache_hit":0,"new_prefill":56286,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":126340,"ongoing_decode_tokens":0,"pending_prefill_tokens":126340,"num_requests":0,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":18432,"new_prefill":37854,"score_linear":107908.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1214,"cache_hit":0,"new_prefill":56286,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":60004,"ongoing_decode_tokens":60004,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1246,"cache_hit":7680,"new_prefill":48606,"score_linear":52324.0,"score_lmetric":48606},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":106951,"ongoing_decode_tokens":106951,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1047,"cache_hit":7680,"new_prefill":48606,"score_linear":99271.0,"score_lmetric":97212}],"chosen_score_linear":107908.0,"chosen_score_lmetric":0,"t_first_token":348843.375252419,"t_first_token_unix":1779779708.3891966,"t_done":348883.402418915,"t_done_unix":1779779748.4163623},{"request_id":"1397223:1:1397223:734","session_id":"1397223","input_length":13134,"t_proxy_recv":348865.875399837,"t_decision_unix":1779779730.8893409,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":13134,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1519,"cache_hit":0,"new_prefill":13134,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1620,"cache_hit":0,"new_prefill":13134,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":38288,"ongoing_decode_tokens":21241,"pending_prefill_tokens":663,"num_requests":2,"active_p_offloads":0,"cached_blocks":1983,"cache_hit":0,"new_prefill":13134,"score_linear":38288.0,"score_lmetric":27594},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1940,"cache_hit":0,"new_prefill":13134,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":246162,"pending_prefill_tokens":201,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":13134,"score_linear":296027.0,"score_lmetric":40005},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":135124,"ongoing_decode_tokens":0,"pending_prefill_tokens":33748,"num_requests":2,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":13134,"score_linear":135124.0,"score_lmetric":93764},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":145452,"ongoing_decode_tokens":23641,"pending_prefill_tokens":59859,"num_requests":2,"active_p_offloads":0,"cached_blocks":1348,"cache_hit":0,"new_prefill":13134,"score_linear":145452.0,"score_lmetric":145986},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":101396,"ongoing_decode_tokens":101396,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1287,"cache_hit":0,"new_prefill":13134,"score_linear":101396.0,"score_lmetric":13134}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348866.970566492,"t_first_token_unix":1779779731.9845102,"t_done":348884.136911465,"t_done_unix":1779779749.1508555},{"request_id":"1355484:16:1401320:758","session_id":"1355484","input_length":32332,"t_proxy_recv":348880.459240949,"t_decision_unix":1779779745.4731824,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":31232,"affinity_cache_ratio":0.9659779784733391,"affinity_num_requests":0,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"cache_hit":31232,"estimated_new_tokens":1100,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1530,"cache_hit":31232,"new_prefill":1100,"score_linear":-31232.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":93040,"ongoing_decode_tokens":0,"pending_prefill_tokens":92528,"num_requests":1,"active_p_offloads":0,"cached_blocks":1620,"cache_hit":0,"new_prefill":32332,"score_linear":93040.0,"score_lmetric":124860},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":124097,"ongoing_decode_tokens":26440,"pending_prefill_tokens":12153,"num_requests":2,"active_p_offloads":0,"cached_blocks":2036,"cache_hit":0,"new_prefill":32332,"score_linear":124097.0,"score_lmetric":88970},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76392,"ongoing_decode_tokens":13134,"pending_prefill_tokens":12058,"num_requests":2,"active_p_offloads":0,"cached_blocks":1940,"cache_hit":0,"new_prefill":32332,"score_linear":76392.0,"score_lmetric":88780},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296954,"ongoing_decode_tokens":119822,"pending_prefill_tokens":1004,"num_requests":3,"active_p_offloads":0,"cached_blocks":2489,"cache_hit":0,"new_prefill":32332,"score_linear":296954.0,"score_lmetric":100008},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":203233,"ongoing_decode_tokens":0,"pending_prefill_tokens":51169,"num_requests":3,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":32332,"score_linear":203233.0,"score_lmetric":250503},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":145452,"ongoing_decode_tokens":23641,"pending_prefill_tokens":59859,"num_requests":2,"active_p_offloads":0,"cached_blocks":1348,"cache_hit":12800,"new_prefill":19532,"score_linear":132652.0,"score_lmetric":158782},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":66493,"ongoing_decode_tokens":0,"pending_prefill_tokens":13757,"num_requests":2,"active_p_offloads":0,"cached_blocks":1342,"cache_hit":12800,"new_prefill":19532,"score_linear":53693.0,"score_lmetric":66578}],"chosen_score_linear":-31232.0,"chosen_score_lmetric":0,"t_first_token":348880.724537788,"t_first_token_unix":1779779745.738481,"t_done":348884.59988207,"t_done_unix":1779779749.613825},{"request_id":"1253743:8:1398344:743","session_id":"1253743","input_length":63258,"t_proxy_recv":348869.912456101,"t_decision_unix":1779779734.9263973,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":51200,"affinity_cache_ratio":0.8093837933542003,"affinity_num_requests":1,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"cache_hit":51200,"estimated_new_tokens":12058,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":34437,"ongoing_decode_tokens":3213,"pending_prefill_tokens":2552,"num_requests":2,"active_p_offloads":0,"cached_blocks":1519,"cache_hit":0,"new_prefill":63258,"score_linear":34437.0,"score_lmetric":131620},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":93040,"ongoing_decode_tokens":0,"pending_prefill_tokens":92528,"num_requests":1,"active_p_offloads":0,"cached_blocks":1620,"cache_hit":4608,"new_prefill":58650,"score_linear":88432.0,"score_lmetric":151178},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":27286,"ongoing_decode_tokens":16550,"pending_prefill_tokens":496,"num_requests":2,"active_p_offloads":0,"cached_blocks":2003,"cache_hit":0,"new_prefill":63258,"score_linear":27286.0,"score_lmetric":127508},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":13134,"ongoing_decode_tokens":13134,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1940,"cache_hit":51200,"new_prefill":12058,"score_linear":-38066.0,"score_lmetric":12058},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":246162,"pending_prefill_tokens":201,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":63258,"score_linear":296027.0,"score_lmetric":190377},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":135124,"ongoing_decode_tokens":0,"pending_prefill_tokens":33748,"num_requests":2,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":17408,"new_prefill":45850,"score_linear":117716.0,"score_lmetric":159196},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":145452,"ongoing_decode_tokens":23641,"pending_prefill_tokens":59859,"num_requests":2,"active_p_offloads":0,"cached_blocks":1348,"cache_hit":0,"new_prefill":63258,"score_linear":145452.0,"score_lmetric":246234},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":3891,"ongoing_decode_tokens":3891,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1289,"cache_hit":0,"new_prefill":63258,"score_linear":3891.0,"score_lmetric":63258}],"chosen_score_linear":-38066.0,"chosen_score_lmetric":12058,"t_first_token":348884.128933501,"t_first_token_unix":1779779749.1428776,"t_done":348885.183279004,"t_done_unix":1779779750.1972222},{"request_id":"1400379:2:1401201:757","session_id":"1400379","input_length":3863,"t_proxy_recv":348880.014029263,"t_decision_unix":1779779745.0279708,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":3584,"affinity_cache_ratio":0.92777633963241,"affinity_num_requests":1,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"cache_hit":3584,"estimated_new_tokens":279,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1530,"cache_hit":0,"new_prefill":3863,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":93040,"ongoing_decode_tokens":0,"pending_prefill_tokens":92528,"num_requests":1,"active_p_offloads":0,"cached_blocks":1620,"cache_hit":0,"new_prefill":3863,"score_linear":93040.0,"score_lmetric":96391},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":124097,"ongoing_decode_tokens":26440,"pending_prefill_tokens":12153,"num_requests":2,"active_p_offloads":0,"cached_blocks":2036,"cache_hit":0,"new_prefill":3863,"score_linear":124097.0,"score_lmetric":32032},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76392,"ongoing_decode_tokens":13134,"pending_prefill_tokens":12058,"num_requests":2,"active_p_offloads":0,"cached_blocks":1940,"cache_hit":0,"new_prefill":3863,"score_linear":76392.0,"score_lmetric":31842},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296954,"ongoing_decode_tokens":119822,"pending_prefill_tokens":1004,"num_requests":3,"active_p_offloads":0,"cached_blocks":2489,"cache_hit":0,"new_prefill":3863,"score_linear":296954.0,"score_lmetric":14601},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":203233,"ongoing_decode_tokens":0,"pending_prefill_tokens":51169,"num_requests":3,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":3863,"score_linear":203233.0,"score_lmetric":165096},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":145452,"ongoing_decode_tokens":23641,"pending_prefill_tokens":59859,"num_requests":2,"active_p_offloads":0,"cached_blocks":1348,"cache_hit":0,"new_prefill":3863,"score_linear":145452.0,"score_lmetric":127444},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":62630,"ongoing_decode_tokens":0,"pending_prefill_tokens":13478,"num_requests":1,"active_p_offloads":0,"cached_blocks":1342,"cache_hit":3584,"new_prefill":279,"score_linear":59046.0,"score_lmetric":13757}],"chosen_score_linear":59046.0,"chosen_score_lmetric":13757,"t_first_token":348884.47384153,"t_first_token_unix":1779779749.4877846,"t_done":348885.190129061,"t_done_unix":1779779750.2040725},{"request_id":"1397840:2:1400216:749","session_id":"1397840","input_length":26440,"t_proxy_recv":348876.375966733,"t_decision_unix":1779779741.3899076,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":16384,"affinity_cache_ratio":0.6196671709531013,"affinity_num_requests":0,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":16384,"estimated_new_tokens":10056,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1530,"cache_hit":0,"new_prefill":26440,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":93040,"ongoing_decode_tokens":0,"pending_prefill_tokens":92528,"num_requests":1,"active_p_offloads":0,"cached_blocks":1620,"cache_hit":0,"new_prefill":26440,"score_linear":93040.0,"score_lmetric":118968},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2036,"cache_hit":16384,"new_prefill":10056,"score_linear":-16384.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76392,"ongoing_decode_tokens":13134,"pending_prefill_tokens":12058,"num_requests":2,"active_p_offloads":0,"cached_blocks":1940,"cache_hit":0,"new_prefill":26440,"score_linear":76392.0,"score_lmetric":76996},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":246162,"pending_prefill_tokens":201,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":26440,"score_linear":296027.0,"score_lmetric":79923},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":135124,"ongoing_decode_tokens":0,"pending_prefill_tokens":33748,"num_requests":2,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":26440,"score_linear":135124.0,"score_lmetric":120376},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":145452,"ongoing_decode_tokens":23641,"pending_prefill_tokens":59859,"num_requests":2,"active_p_offloads":0,"cached_blocks":1348,"cache_hit":0,"new_prefill":26440,"score_linear":145452.0,"score_lmetric":172598},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":4991,"ongoing_decode_tokens":4991,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1311,"cache_hit":0,"new_prefill":26440,"score_linear":4991.0,"score_lmetric":26440}],"chosen_score_linear":-16384.0,"chosen_score_lmetric":0,"t_first_token":348878.083943536,"t_first_token_unix":1779779743.0978868,"t_done":348885.249521726,"t_done_unix":1779779750.2634654},{"request_id":"1397840:3:1402568:761","session_id":"1397840","input_length":28603,"t_proxy_recv":348885.257396959,"t_decision_unix":1779779750.2713385,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":26112,"affinity_cache_ratio":0.9129112330874384,"affinity_num_requests":1,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":26112,"estimated_new_tokens":2491,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1533,"cache_hit":0,"new_prefill":28603,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":93040,"ongoing_decode_tokens":0,"pending_prefill_tokens":92528,"num_requests":1,"active_p_offloads":0,"cached_blocks":1620,"cache_hit":0,"new_prefill":28603,"score_linear":93040.0,"score_lmetric":121131},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":97657,"ongoing_decode_tokens":97657,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2055,"cache_hit":26112,"new_prefill":2491,"score_linear":71545.0,"score_lmetric":2491},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1989,"cache_hit":0,"new_prefill":28603,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":302125,"ongoing_decode_tokens":113401,"pending_prefill_tokens":5428,"num_requests":3,"active_p_offloads":0,"cached_blocks":2564,"cache_hit":0,"new_prefill":28603,"score_linear":302125.0,"score_lmetric":102093},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":203233,"ongoing_decode_tokens":0,"pending_prefill_tokens":51169,"num_requests":3,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":28603,"score_linear":203233.0,"score_lmetric":239316},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":145452,"ongoing_decode_tokens":23641,"pending_prefill_tokens":59859,"num_requests":2,"active_p_offloads":0,"cached_blocks":1348,"cache_hit":0,"new_prefill":28603,"score_linear":145452.0,"score_lmetric":176924},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":75540,"ongoing_decode_tokens":75540,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1342,"cache_hit":0,"new_prefill":28603,"score_linear":75540.0,"score_lmetric":57206}],"chosen_score_linear":71545.0,"chosen_score_lmetric":2491,"t_first_token":348885.780061299,"t_first_token_unix":1779779750.7940052,"t_done":348886.101074952,"t_done_unix":1779779751.1150188},{"request_id":"1366128:20:1400446:752","session_id":"1366128","input_length":97657,"t_proxy_recv":348877.144792168,"t_decision_unix":1779779742.1587334,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":85504,"affinity_cache_ratio":0.8755542357434695,"affinity_num_requests":1,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"cache_hit":85504,"estimated_new_tokens":12153,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":31469,"ongoing_decode_tokens":31469,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1530,"cache_hit":0,"new_prefill":97657,"score_linear":31469.0,"score_lmetric":97657},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":93040,"ongoing_decode_tokens":0,"pending_prefill_tokens":92528,"num_requests":1,"active_p_offloads":0,"cached_blocks":1620,"cache_hit":0,"new_prefill":97657,"score_linear":93040.0,"score_lmetric":190185},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":26440,"ongoing_decode_tokens":0,"pending_prefill_tokens":10056,"num_requests":1,"active_p_offloads":0,"cached_blocks":2036,"cache_hit":85504,"new_prefill":12153,"score_linear":-59064.0,"score_lmetric":22209},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76392,"ongoing_decode_tokens":13134,"pending_prefill_tokens":12058,"num_requests":2,"active_p_offloads":0,"cached_blocks":1940,"cache_hit":0,"new_prefill":97657,"score_linear":76392.0,"score_lmetric":219430},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":246162,"pending_prefill_tokens":201,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":97657,"score_linear":296027.0,"score_lmetric":293574},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":135124,"ongoing_decode_tokens":0,"pending_prefill_tokens":33748,"num_requests":2,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":97657,"score_linear":135124.0,"score_lmetric":262810},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":145452,"ongoing_decode_tokens":23641,"pending_prefill_tokens":59859,"num_requests":2,"active_p_offloads":0,"cached_blocks":1348,"cache_hit":0,"new_prefill":97657,"score_linear":145452.0,"score_lmetric":315032},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":3707,"ongoing_decode_tokens":3707,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1320,"cache_hit":0,"new_prefill":97657,"score_linear":3707.0,"score_lmetric":97657}],"chosen_score_linear":-59064.0,"chosen_score_lmetric":22209,"t_first_token":348884.94188509,"t_first_token_unix":1779779749.955829,"t_done":348886.395160924,"t_done_unix":1779779751.4091058},{"request_id":"1366128:21:1402408:760","session_id":"1366128","input_length":97814,"t_proxy_recv":348886.411713548,"t_decision_unix":1779779751.425655,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":97280,"affinity_cache_ratio":0.9945406588013985,"affinity_num_requests":0,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":97280,"estimated_new_tokens":534,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":11216,"ongoing_decode_tokens":0,"pending_prefill_tokens":11216,"num_requests":1,"active_p_offloads":0,"cached_blocks":1533,"cache_hit":0,"new_prefill":97814,"score_linear":11216.0,"score_lmetric":109030},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":93040,"ongoing_decode_tokens":0,"pending_prefill_tokens":92528,"num_requests":1,"active_p_offloads":0,"cached_blocks":1620,"cache_hit":0,"new_prefill":97814,"score_linear":93040.0,"score_lmetric":190342},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2082,"cache_hit":97280,"new_prefill":534,"score_linear":-97280.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1989,"cache_hit":0,"new_prefill":97814,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":302125,"ongoing_decode_tokens":113401,"pending_prefill_tokens":5428,"num_requests":3,"active_p_offloads":0,"cached_blocks":2564,"cache_hit":0,"new_prefill":97814,"score_linear":302125.0,"score_lmetric":309726},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":203233,"ongoing_decode_tokens":0,"pending_prefill_tokens":51169,"num_requests":3,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":97814,"score_linear":203233.0,"score_lmetric":446949},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":145452,"ongoing_decode_tokens":145452,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1348,"cache_hit":0,"new_prefill":97814,"score_linear":145452.0,"score_lmetric":195628},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":75540,"ongoing_decode_tokens":75540,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1342,"cache_hit":0,"new_prefill":97814,"score_linear":75540.0,"score_lmetric":195628}],"chosen_score_linear":-97280.0,"chosen_score_lmetric":0,"t_first_token":348886.614454479,"t_first_token_unix":1779779751.628398,"t_done":348887.000052866,"t_done_unix":1779779752.0139966},{"request_id":"1313181:7:1327489:383","session_id":"1313181","input_length":121811,"t_proxy_recv":348851.78404166,"t_decision_unix":1779779716.7979827,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":61952,"affinity_cache_ratio":0.5085911781366215,"affinity_num_requests":1,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":61952,"estimated_new_tokens":59859,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1502,"cache_hit":24064,"new_prefill":97747,"score_linear":-24064.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":92082,"ongoing_decode_tokens":0,"pending_prefill_tokens":73138,"num_requests":2,"active_p_offloads":0,"cached_blocks":1400,"cache_hit":0,"new_prefill":121811,"score_linear":92082.0,"score_lmetric":389898},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":84897,"ongoing_decode_tokens":84897,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1979,"cache_hit":0,"new_prefill":121811,"score_linear":84897.0,"score_lmetric":121811},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1927,"cache_hit":0,"new_prefill":121811,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":182626,"pending_prefill_tokens":761,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":121811,"score_linear":296027.0,"score_lmetric":367716},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":121811,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":23641,"ongoing_decode_tokens":23641,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1348,"cache_hit":61952,"new_prefill":59859,"score_linear":-38311.0,"score_lmetric":59859},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":147211,"ongoing_decode_tokens":147211,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1117,"cache_hit":12800,"new_prefill":109011,"score_linear":134411.0,"score_lmetric":218022}],"chosen_score_linear":-38311.0,"chosen_score_lmetric":59859,"t_first_token":348885.620934208,"t_first_token_unix":1779779750.6348772,"t_done":348887.160201136,"t_done_unix":1779779752.1741443},{"request_id":"1400379:3:1402972:763","session_id":"1400379","input_length":11216,"t_proxy_recv":348886.219115318,"t_decision_unix":1779779751.2330568,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":7,"chosen_idx":0,"affinity_cache_hit":3584,"affinity_cache_ratio":0.3195435092724679,"affinity_num_requests":2,"avg_num_requests":1.5,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":11216,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1533,"cache_hit":0,"new_prefill":11216,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":93040,"ongoing_decode_tokens":0,"pending_prefill_tokens":92528,"num_requests":1,"active_p_offloads":0,"cached_blocks":1620,"cache_hit":0,"new_prefill":11216,"score_linear":93040.0,"score_lmetric":103744},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":97657,"ongoing_decode_tokens":97657,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2059,"cache_hit":0,"new_prefill":11216,"score_linear":97657.0,"score_lmetric":11216},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1989,"cache_hit":0,"new_prefill":11216,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":302125,"ongoing_decode_tokens":113401,"pending_prefill_tokens":5428,"num_requests":3,"active_p_offloads":0,"cached_blocks":2564,"cache_hit":0,"new_prefill":11216,"score_linear":302125.0,"score_lmetric":49932},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":203233,"ongoing_decode_tokens":0,"pending_prefill_tokens":51169,"num_requests":3,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":11216,"score_linear":203233.0,"score_lmetric":187155},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":145452,"ongoing_decode_tokens":145452,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1348,"cache_hit":0,"new_prefill":11216,"score_linear":145452.0,"score_lmetric":22432},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":75540,"ongoing_decode_tokens":75540,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1342,"cache_hit":3584,"new_prefill":7632,"score_linear":71956.0,"score_lmetric":15264}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348887.107935728,"t_first_token_unix":1779779752.121879,"t_done":348887.35132159,"t_done_unix":1779779752.3652647},{"request_id":"1403188:1:1403188:764","session_id":"1403188","input_length":8848,"t_proxy_recv":348887.119807699,"t_decision_unix":1779779752.1337492,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.5,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":8848,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":11216,"ongoing_decode_tokens":11216,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1533,"cache_hit":0,"new_prefill":8848,"score_linear":11216.0,"score_lmetric":8848},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":93040,"ongoing_decode_tokens":0,"pending_prefill_tokens":92528,"num_requests":1,"active_p_offloads":0,"cached_blocks":1620,"cache_hit":0,"new_prefill":8848,"score_linear":93040.0,"score_lmetric":101376},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2083,"cache_hit":0,"new_prefill":8848,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1989,"cache_hit":0,"new_prefill":8848,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":302125,"ongoing_decode_tokens":113401,"pending_prefill_tokens":5428,"num_requests":3,"active_p_offloads":0,"cached_blocks":2564,"cache_hit":0,"new_prefill":8848,"score_linear":302125.0,"score_lmetric":42828},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":203233,"ongoing_decode_tokens":0,"pending_prefill_tokens":51169,"num_requests":3,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":8848,"score_linear":203233.0,"score_lmetric":180051},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":145452,"ongoing_decode_tokens":145452,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1348,"cache_hit":0,"new_prefill":8848,"score_linear":145452.0,"score_lmetric":17696},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":75540,"ongoing_decode_tokens":75540,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1342,"cache_hit":0,"new_prefill":8848,"score_linear":75540.0,"score_lmetric":17696}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348887.787429828,"t_first_token_unix":1779779752.8013735,"t_done":348888.361966476,"t_done_unix":1779779753.3759098},{"request_id":"1398213:3:1401507:759","session_id":"1398213","input_length":12910,"t_proxy_recv":348881.10295593,"t_decision_unix":1779779746.1168976,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":11264,"affinity_cache_ratio":0.8725019364833463,"affinity_num_requests":2,"avg_num_requests":2.0,"fallback_score":null,"tie_break_used":false,"cache_hit":11264,"estimated_new_tokens":1646,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":32332,"ongoing_decode_tokens":32332,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1530,"cache_hit":0,"new_prefill":12910,"score_linear":32332.0,"score_lmetric":12910},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":93040,"ongoing_decode_tokens":0,"pending_prefill_tokens":92528,"num_requests":1,"active_p_offloads":0,"cached_blocks":1620,"cache_hit":0,"new_prefill":12910,"score_linear":93040.0,"score_lmetric":105438},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":124097,"ongoing_decode_tokens":26440,"pending_prefill_tokens":12153,"num_requests":2,"active_p_offloads":0,"cached_blocks":2036,"cache_hit":0,"new_prefill":12910,"score_linear":124097.0,"score_lmetric":50126},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76392,"ongoing_decode_tokens":13134,"pending_prefill_tokens":12058,"num_requests":2,"active_p_offloads":0,"cached_blocks":1940,"cache_hit":0,"new_prefill":12910,"score_linear":76392.0,"score_lmetric":49936},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296954,"ongoing_decode_tokens":119822,"pending_prefill_tokens":1004,"num_requests":3,"active_p_offloads":0,"cached_blocks":2489,"cache_hit":0,"new_prefill":12910,"score_linear":296954.0,"score_lmetric":41742},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":203233,"ongoing_decode_tokens":0,"pending_prefill_tokens":51169,"num_requests":3,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":12910,"score_linear":203233.0,"score_lmetric":192237},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":145452,"ongoing_decode_tokens":23641,"pending_prefill_tokens":59859,"num_requests":2,"active_p_offloads":0,"cached_blocks":1348,"cache_hit":0,"new_prefill":12910,"score_linear":145452.0,"score_lmetric":145538},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":66493,"ongoing_decode_tokens":0,"pending_prefill_tokens":13757,"num_requests":2,"active_p_offloads":0,"cached_blocks":1342,"cache_hit":11264,"new_prefill":1646,"score_linear":55229.0,"score_lmetric":30806}],"chosen_score_linear":55229.0,"chosen_score_lmetric":30806,"t_first_token":348884.472076858,"t_first_token_unix":1779779749.4860203,"t_done":348888.977071436,"t_done_unix":1779779753.9910145},{"request_id":"1366128:22:1403837:766","session_id":"1366128","input_length":97917,"t_proxy_recv":348889.154467307,"t_decision_unix":1779779754.1684089,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":97792,"affinity_cache_ratio":0.9987234086011622,"affinity_num_requests":0,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":97792,"estimated_new_tokens":125,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1554,"cache_hit":0,"new_prefill":97917,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":137188,"ongoing_decode_tokens":0,"pending_prefill_tokens":95204,"num_requests":2,"active_p_offloads":0,"cached_blocks":1620,"cache_hit":0,"new_prefill":97917,"score_linear":137188.0,"score_lmetric":386242},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2083,"cache_hit":97792,"new_prefill":125,"score_linear":-97792.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2006,"cache_hit":0,"new_prefill":97917,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":302125,"ongoing_decode_tokens":113401,"pending_prefill_tokens":5428,"num_requests":3,"active_p_offloads":0,"cached_blocks":2564,"cache_hit":0,"new_prefill":97917,"score_linear":302125.0,"score_lmetric":310035},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":203233,"ongoing_decode_tokens":101803,"pending_prefill_tokens":22582,"num_requests":3,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":97917,"score_linear":203233.0,"score_lmetric":361497},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":85659,"ongoing_decode_tokens":85659,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1380,"cache_hit":0,"new_prefill":97917,"score_linear":85659.0,"score_lmetric":195834},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":62630,"ongoing_decode_tokens":62630,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1346,"cache_hit":0,"new_prefill":97917,"score_linear":62630.0,"score_lmetric":97917}],"chosen_score_linear":-97792.0,"chosen_score_lmetric":0,"t_first_token":348889.344191288,"t_first_token_unix":1779779754.3581345,"t_done":348889.413197164,"t_done_unix":1779779754.42714},{"request_id":"1364090:5:1401054:756","session_id":"1364090","input_length":62630,"t_proxy_recv":348879.522031274,"t_decision_unix":1779779744.5359726,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":49152,"affinity_cache_ratio":0.7847996167970621,"affinity_num_requests":0,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"cache_hit":49152,"estimated_new_tokens":13478,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1530,"cache_hit":7680,"new_prefill":54950,"score_linear":-7680.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":93040,"ongoing_decode_tokens":0,"pending_prefill_tokens":92528,"num_requests":1,"active_p_offloads":0,"cached_blocks":1620,"cache_hit":0,"new_prefill":62630,"score_linear":93040.0,"score_lmetric":155158},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":124097,"ongoing_decode_tokens":26440,"pending_prefill_tokens":12153,"num_requests":2,"active_p_offloads":0,"cached_blocks":2036,"cache_hit":0,"new_prefill":62630,"score_linear":124097.0,"score_lmetric":149566},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76392,"ongoing_decode_tokens":13134,"pending_prefill_tokens":12058,"num_requests":2,"active_p_offloads":0,"cached_blocks":1940,"cache_hit":0,"new_prefill":62630,"score_linear":76392.0,"score_lmetric":149376},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296954,"ongoing_decode_tokens":119822,"pending_prefill_tokens":1004,"num_requests":3,"active_p_offloads":0,"cached_blocks":2489,"cache_hit":7680,"new_prefill":54950,"score_linear":289274.0,"score_lmetric":167862},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":203233,"ongoing_decode_tokens":0,"pending_prefill_tokens":51169,"num_requests":3,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":62630,"score_linear":203233.0,"score_lmetric":341397},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":145452,"ongoing_decode_tokens":23641,"pending_prefill_tokens":59859,"num_requests":2,"active_p_offloads":0,"cached_blocks":1348,"cache_hit":7680,"new_prefill":54950,"score_linear":137772.0,"score_lmetric":229618},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1342,"cache_hit":49152,"new_prefill":13478,"score_linear":-49152.0,"score_lmetric":0}],"chosen_score_linear":-49152.0,"chosen_score_lmetric":0,"t_first_token":348884.351916586,"t_first_token_unix":1779779749.36586,"t_done":348889.805327363,"t_done_unix":1779779754.8192708},{"request_id":"1403989:1:1403989:767","session_id":"1403989","input_length":1954,"t_proxy_recv":348889.619549898,"t_decision_unix":1779779754.6334918,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":1954,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1554,"cache_hit":0,"new_prefill":1954,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":137188,"ongoing_decode_tokens":0,"pending_prefill_tokens":95204,"num_requests":2,"active_p_offloads":0,"cached_blocks":1620,"cache_hit":0,"new_prefill":1954,"score_linear":137188.0,"score_lmetric":194316},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2083,"cache_hit":0,"new_prefill":1954,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2006,"cache_hit":0,"new_prefill":1954,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":302125,"ongoing_decode_tokens":113401,"pending_prefill_tokens":5428,"num_requests":3,"active_p_offloads":0,"cached_blocks":2564,"cache_hit":0,"new_prefill":1954,"score_linear":302125.0,"score_lmetric":22146},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":203233,"ongoing_decode_tokens":101803,"pending_prefill_tokens":22582,"num_requests":3,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":1954,"score_linear":203233.0,"score_lmetric":73608},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":85659,"ongoing_decode_tokens":85659,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1380,"cache_hit":0,"new_prefill":1954,"score_linear":85659.0,"score_lmetric":3908},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":62630,"ongoing_decode_tokens":62630,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1346,"cache_hit":0,"new_prefill":1954,"score_linear":62630.0,"score_lmetric":1954}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348889.746156333,"t_first_token_unix":1779779754.7601006,"t_done":348890.098671145,"t_done_unix":1779779755.1126268},{"request_id":"1363943:9:1390468:707","session_id":"1363943","input_length":23641,"t_proxy_recv":348848.709438915,"t_decision_unix":1779779713.7233806,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":21504,"affinity_cache_ratio":0.9096061926314454,"affinity_num_requests":1,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":21504,"estimated_new_tokens":2137,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":28382,"ongoing_decode_tokens":28382,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1495,"cache_hit":8704,"new_prefill":14937,"score_linear":19678.0,"score_lmetric":14937},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":92082,"ongoing_decode_tokens":0,"pending_prefill_tokens":73138,"num_requests":2,"active_p_offloads":0,"cached_blocks":1400,"cache_hit":0,"new_prefill":23641,"score_linear":92082.0,"score_lmetric":193558},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1979,"cache_hit":0,"new_prefill":23641,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1926,"cache_hit":0,"new_prefill":23641,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":182626,"pending_prefill_tokens":761,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":4096,"new_prefill":19545,"score_linear":291931.0,"score_lmetric":60918},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":28307,"ongoing_decode_tokens":0,"pending_prefill_tokens":25747,"num_requests":1,"active_p_offloads":0,"cached_blocks":1538,"cache_hit":0,"new_prefill":23641,"score_linear":28307.0,"score_lmetric":49388},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":61543,"ongoing_decode_tokens":61543,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1347,"cache_hit":21504,"new_prefill":2137,"score_linear":40039.0,"score_lmetric":2137},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":98267,"ongoing_decode_tokens":98267,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1117,"cache_hit":4096,"new_prefill":19545,"score_linear":94171.0,"score_lmetric":19545}],"chosen_score_linear":40039.0,"chosen_score_lmetric":2137,"t_first_token":348849.071419973,"t_first_token_unix":1779779714.0853636,"t_done":348890.905971425,"t_done_unix":1779779755.9199147},{"request_id":"1253743:8:1404284:769","session_id":"1253743","input_length":63643,"t_proxy_recv":348890.643997382,"t_decision_unix":1779779755.657939,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":62976,"affinity_cache_ratio":0.9895196643778578,"affinity_num_requests":0,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":62976,"estimated_new_tokens":667,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1554,"cache_hit":0,"new_prefill":63643,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":137188,"ongoing_decode_tokens":0,"pending_prefill_tokens":95204,"num_requests":2,"active_p_offloads":0,"cached_blocks":1620,"cache_hit":4608,"new_prefill":59035,"score_linear":132580.0,"score_lmetric":308478},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2083,"cache_hit":0,"new_prefill":63643,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2009,"cache_hit":62976,"new_prefill":667,"score_linear":-62976.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":302125,"ongoing_decode_tokens":113401,"pending_prefill_tokens":5428,"num_requests":3,"active_p_offloads":0,"cached_blocks":2564,"cache_hit":0,"new_prefill":63643,"score_linear":302125.0,"score_lmetric":207213},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":203233,"ongoing_decode_tokens":101803,"pending_prefill_tokens":22582,"num_requests":3,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":17408,"new_prefill":46235,"score_linear":185825.0,"score_lmetric":206451},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":85659,"ongoing_decode_tokens":85659,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1380,"cache_hit":0,"new_prefill":63643,"score_linear":85659.0,"score_lmetric":127286},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":0,"new_prefill":63643,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-62976.0,"chosen_score_lmetric":0,"t_first_token":348890.867240465,"t_first_token_unix":1779779755.8811839,"t_done":348891.50236224,"t_done_unix":1779779756.5163057},{"request_id":"1313181:7:1382851:662","session_id":"1313181","input_length":62018,"t_proxy_recv":348887.168304009,"t_decision_unix":1779779752.1822457,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":61952,"affinity_cache_ratio":0.9989357928343384,"affinity_num_requests":1,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":61952,"estimated_new_tokens":66,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":11216,"ongoing_decode_tokens":11216,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1533,"cache_hit":24064,"new_prefill":37954,"score_linear":-12848.0,"score_lmetric":37954},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":93040,"ongoing_decode_tokens":0,"pending_prefill_tokens":92528,"num_requests":1,"active_p_offloads":0,"cached_blocks":1620,"cache_hit":0,"new_prefill":62018,"score_linear":93040.0,"score_lmetric":154546},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2083,"cache_hit":0,"new_prefill":62018,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":8848,"ongoing_decode_tokens":0,"pending_prefill_tokens":8848,"num_requests":1,"active_p_offloads":0,"cached_blocks":1989,"cache_hit":0,"new_prefill":62018,"score_linear":8848.0,"score_lmetric":70866},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":302125,"ongoing_decode_tokens":113401,"pending_prefill_tokens":5428,"num_requests":3,"active_p_offloads":0,"cached_blocks":2564,"cache_hit":0,"new_prefill":62018,"score_linear":302125.0,"score_lmetric":202338},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":203233,"ongoing_decode_tokens":0,"pending_prefill_tokens":51169,"num_requests":3,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":62018,"score_linear":203233.0,"score_lmetric":339561},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":23641,"ongoing_decode_tokens":23641,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1380,"cache_hit":61952,"new_prefill":66,"score_linear":-38311.0,"score_lmetric":66},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":75540,"ongoing_decode_tokens":75540,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1342,"cache_hit":12800,"new_prefill":49218,"score_linear":62740.0,"score_lmetric":98436}],"chosen_score_linear":-38311.0,"chosen_score_lmetric":66,"t_first_token":348887.256686412,"t_first_token_unix":1779779752.27063,"t_done":348891.528096383,"t_done_unix":1779779756.5420399},{"request_id":"1405199:1:1405199:776","session_id":"1405199","input_length":11503,"t_proxy_recv":348893.863165819,"t_decision_unix":1779779758.8771071,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.875,"fallback_score":11503,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":11503,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":21684,"ongoing_decode_tokens":21684,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1554,"cache_hit":0,"new_prefill":11503,"score_linear":21684.0,"score_lmetric":11503},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":137188,"ongoing_decode_tokens":0,"pending_prefill_tokens":95204,"num_requests":2,"active_p_offloads":0,"cached_blocks":1620,"cache_hit":0,"new_prefill":11503,"score_linear":137188.0,"score_lmetric":213414},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":145650,"ongoing_decode_tokens":0,"pending_prefill_tokens":26354,"num_requests":3,"active_p_offloads":0,"cached_blocks":2083,"cache_hit":0,"new_prefill":11503,"score_linear":145650.0,"score_lmetric":113571},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":32944,"ongoing_decode_tokens":0,"pending_prefill_tokens":32944,"num_requests":1,"active_p_offloads":0,"cached_blocks":2010,"cache_hit":0,"new_prefill":11503,"score_linear":32944.0,"score_lmetric":44447},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":302125,"ongoing_decode_tokens":113401,"pending_prefill_tokens":5428,"num_requests":3,"active_p_offloads":0,"cached_blocks":2564,"cache_hit":0,"new_prefill":11503,"score_linear":302125.0,"score_lmetric":50793},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":203233,"ongoing_decode_tokens":135124,"pending_prefill_tokens":17421,"num_requests":3,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":11503,"score_linear":203233.0,"score_lmetric":86772},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":24442,"ongoing_decode_tokens":24442,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1385,"cache_hit":0,"new_prefill":11503,"score_linear":24442.0,"score_lmetric":11503},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":124090,"ongoing_decode_tokens":0,"pending_prefill_tokens":111290,"num_requests":1,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":0,"new_prefill":11503,"score_linear":124090.0,"score_lmetric":122793}],"chosen_score_linear":24442.0,"chosen_score_lmetric":11503,"t_first_token":348894.789801964,"t_first_token_unix":1779779759.8037453,"t_done":348895.150670836,"t_done_unix":1779779760.1646142},{"request_id":"1363943:10:1401002:755","session_id":"1363943","input_length":24442,"t_proxy_recv":348890.911127352,"t_decision_unix":1779779755.925069,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":24064,"affinity_cache_ratio":0.9845348171180754,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":24064,"estimated_new_tokens":378,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1554,"cache_hit":8704,"new_prefill":15738,"score_linear":-8704.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":137188,"ongoing_decode_tokens":0,"pending_prefill_tokens":95204,"num_requests":2,"active_p_offloads":0,"cached_blocks":1620,"cache_hit":0,"new_prefill":24442,"score_linear":137188.0,"score_lmetric":239292},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2083,"cache_hit":0,"new_prefill":24442,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":63643,"ongoing_decode_tokens":63643,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2009,"cache_hit":0,"new_prefill":24442,"score_linear":63643.0,"score_lmetric":24442},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":302125,"ongoing_decode_tokens":113401,"pending_prefill_tokens":5428,"num_requests":3,"active_p_offloads":0,"cached_blocks":2564,"cache_hit":4096,"new_prefill":20346,"score_linear":298029.0,"score_lmetric":77322},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":203233,"ongoing_decode_tokens":135124,"pending_prefill_tokens":17421,"num_requests":3,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":24442,"score_linear":203233.0,"score_lmetric":125589},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":62018,"ongoing_decode_tokens":62018,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1385,"cache_hit":24064,"new_prefill":378,"score_linear":37954.0,"score_lmetric":378},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":4096,"new_prefill":20346,"score_linear":-4096.0,"score_lmetric":0}],"chosen_score_linear":37954.0,"chosen_score_lmetric":378,"t_first_token":348890.988895597,"t_first_token_unix":1779779756.0028389,"t_done":348896.230797929,"t_done_unix":1779779761.2447412},{"request_id":"1340278:3:1384549:676","session_id":"1340278","input_length":49865,"t_proxy_recv":348821.459235091,"t_decision_unix":1779779686.4731765,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":49664,"affinity_cache_ratio":0.9959691166148601,"affinity_num_requests":2,"avg_num_requests":2.0,"fallback_score":null,"tie_break_used":false,"cache_hit":49664,"estimated_new_tokens":201,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":31215,"ongoing_decode_tokens":5989,"pending_prefill_tokens":1674,"num_requests":2,"active_p_offloads":0,"cached_blocks":1440,"cache_hit":0,"new_prefill":49865,"score_linear":31215.0,"score_lmetric":103078},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":15283,"ongoing_decode_tokens":15283,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1284,"cache_hit":0,"new_prefill":49865,"score_linear":15283.0,"score_lmetric":49865},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":161742,"ongoing_decode_tokens":140521,"pending_prefill_tokens":229,"num_requests":3,"active_p_offloads":0,"cached_blocks":1916,"cache_hit":0,"new_prefill":49865,"score_linear":161742.0,"score_lmetric":150282},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":69465,"ongoing_decode_tokens":0,"pending_prefill_tokens":3417,"num_requests":1,"active_p_offloads":0,"cached_blocks":1887,"cache_hit":0,"new_prefill":49865,"score_linear":69465.0,"score_lmetric":53282},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":246162,"ongoing_decode_tokens":0,"pending_prefill_tokens":164754,"num_requests":2,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":49664,"new_prefill":201,"score_linear":196498.0,"score_lmetric":329910},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":78218,"ongoing_decode_tokens":13989,"pending_prefill_tokens":56549,"num_requests":3,"active_p_offloads":0,"cached_blocks":1327,"cache_hit":0,"new_prefill":49865,"score_linear":78218.0,"score_lmetric":319242},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":127253,"ongoing_decode_tokens":20948,"pending_prefill_tokens":44865,"num_requests":2,"active_p_offloads":0,"cached_blocks":1343,"cache_hit":0,"new_prefill":49865,"score_linear":127253.0,"score_lmetric":189460},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":145668,"ongoing_decode_tokens":0,"pending_prefill_tokens":105220,"num_requests":2,"active_p_offloads":0,"cached_blocks":1077,"cache_hit":0,"new_prefill":49865,"score_linear":145668.0,"score_lmetric":310170}],"chosen_score_linear":196498.0,"chosen_score_lmetric":329910,"t_first_token":348882.785749638,"t_first_token_unix":1779779747.7996933,"t_done":348897.547564979,"t_done_unix":1779779762.5615125},{"request_id":"1381387:4:1400724:754","session_id":"1381387","input_length":68109,"t_proxy_recv":348878.252549278,"t_decision_unix":1779779743.266491,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":50688,"affinity_cache_ratio":0.7442188257058538,"affinity_num_requests":2,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"cache_hit":50688,"estimated_new_tokens":17421,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1530,"cache_hit":0,"new_prefill":68109,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":93040,"ongoing_decode_tokens":0,"pending_prefill_tokens":92528,"num_requests":1,"active_p_offloads":0,"cached_blocks":1620,"cache_hit":0,"new_prefill":68109,"score_linear":93040.0,"score_lmetric":160637},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":124097,"ongoing_decode_tokens":26440,"pending_prefill_tokens":12153,"num_requests":2,"active_p_offloads":0,"cached_blocks":2036,"cache_hit":0,"new_prefill":68109,"score_linear":124097.0,"score_lmetric":160524},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76392,"ongoing_decode_tokens":13134,"pending_prefill_tokens":12058,"num_requests":2,"active_p_offloads":0,"cached_blocks":1940,"cache_hit":0,"new_prefill":68109,"score_linear":76392.0,"score_lmetric":160334},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296954,"ongoing_decode_tokens":119822,"pending_prefill_tokens":1004,"num_requests":3,"active_p_offloads":0,"cached_blocks":2489,"cache_hit":0,"new_prefill":68109,"score_linear":296954.0,"score_lmetric":207339},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":135124,"ongoing_decode_tokens":0,"pending_prefill_tokens":33748,"num_requests":2,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":50688,"new_prefill":17421,"score_linear":84436.0,"score_lmetric":102338},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":145452,"ongoing_decode_tokens":23641,"pending_prefill_tokens":59859,"num_requests":2,"active_p_offloads":0,"cached_blocks":1348,"cache_hit":0,"new_prefill":68109,"score_linear":145452.0,"score_lmetric":255936},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":11414,"ongoing_decode_tokens":0,"pending_prefill_tokens":7830,"num_requests":1,"active_p_offloads":0,"cached_blocks":1327,"cache_hit":0,"new_prefill":68109,"score_linear":11414.0,"score_lmetric":75939}],"chosen_score_linear":84436.0,"chosen_score_lmetric":102338,"t_first_token":348894.474365084,"t_first_token_unix":1779779759.4883084,"t_done":348899.416407465,"t_done_unix":1779779764.4303508},{"request_id":"1406966:1:1406966:785","session_id":"1406966","input_length":2115,"t_proxy_recv":348900.113820358,"t_decision_unix":1779779765.1277618,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.375,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":2115,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":154908,"ongoing_decode_tokens":21684,"pending_prefill_tokens":12392,"num_requests":3,"active_p_offloads":0,"cached_blocks":1554,"cache_hit":0,"new_prefill":2115,"score_linear":154908.0,"score_lmetric":43521},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":137188,"ongoing_decode_tokens":93040,"pending_prefill_tokens":2676,"num_requests":2,"active_p_offloads":0,"cached_blocks":1620,"cache_hit":0,"new_prefill":2115,"score_linear":137188.0,"score_lmetric":9582},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":255224,"ongoing_decode_tokens":60212,"pending_prefill_tokens":15812,"num_requests":4,"active_p_offloads":0,"cached_blocks":2083,"cache_hit":0,"new_prefill":2115,"score_linear":255224.0,"score_lmetric":71708},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":32944,"ongoing_decode_tokens":32944,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2010,"cache_hit":0,"new_prefill":2115,"score_linear":32944.0,"score_lmetric":2115},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":302532,"ongoing_decode_tokens":190803,"pending_prefill_tokens":5233,"num_requests":3,"active_p_offloads":0,"cached_blocks":2564,"cache_hit":0,"new_prefill":2115,"score_linear":302532.0,"score_lmetric":22044},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":204632,"ongoing_decode_tokens":204632,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1622,"cache_hit":0,"new_prefill":2115,"score_linear":204632.0,"score_lmetric":6345},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1408,"cache_hit":0,"new_prefill":2115,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":250216,"ongoing_decode_tokens":0,"pending_prefill_tokens":116584,"num_requests":3,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":0,"new_prefill":2115,"score_linear":250216.0,"score_lmetric":356097}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348900.250544439,"t_first_token_unix":1779779765.2644875,"t_done":348900.478392951,"t_done_unix":1779779765.4923363},{"request_id":"1407494:1:1407494:787","session_id":"1407494","input_length":1531,"t_proxy_recv":348901.916453645,"t_decision_unix":1779779766.930395,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.375,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":1531,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":154908,"ongoing_decode_tokens":21684,"pending_prefill_tokens":12392,"num_requests":3,"active_p_offloads":0,"cached_blocks":1554,"cache_hit":0,"new_prefill":1531,"score_linear":154908.0,"score_lmetric":41769},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":137188,"ongoing_decode_tokens":93040,"pending_prefill_tokens":2676,"num_requests":2,"active_p_offloads":0,"cached_blocks":1620,"cache_hit":0,"new_prefill":1531,"score_linear":137188.0,"score_lmetric":8414},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":255224,"ongoing_decode_tokens":60212,"pending_prefill_tokens":15812,"num_requests":4,"active_p_offloads":0,"cached_blocks":2083,"cache_hit":0,"new_prefill":1531,"score_linear":255224.0,"score_lmetric":69372},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":32944,"ongoing_decode_tokens":32944,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2010,"cache_hit":0,"new_prefill":1531,"score_linear":32944.0,"score_lmetric":1531},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":302532,"ongoing_decode_tokens":190803,"pending_prefill_tokens":5233,"num_requests":3,"active_p_offloads":0,"cached_blocks":2564,"cache_hit":0,"new_prefill":1531,"score_linear":302532.0,"score_lmetric":20292},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":204632,"ongoing_decode_tokens":204632,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1622,"cache_hit":0,"new_prefill":1531,"score_linear":204632.0,"score_lmetric":4593},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1412,"cache_hit":0,"new_prefill":1531,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":250216,"ongoing_decode_tokens":0,"pending_prefill_tokens":116584,"num_requests":3,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":0,"new_prefill":1531,"score_linear":250216.0,"score_lmetric":354345}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348902.02461422,"t_first_token_unix":1779779767.0385573,"t_done":348902.291952156,"t_done_unix":1779779767.305895},{"request_id":"1406966:2:1407710:789","session_id":"1406966","input_length":4789,"t_proxy_recv":348902.592661414,"t_decision_unix":1779779767.6066027,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":2048,"affinity_cache_ratio":0.42764669033201086,"affinity_num_requests":0,"avg_num_requests":2.5,"fallback_score":0,"tie_break_used":false,"cache_hit":2048,"estimated_new_tokens":2741,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":154908,"ongoing_decode_tokens":21684,"pending_prefill_tokens":12392,"num_requests":3,"active_p_offloads":0,"cached_blocks":1554,"cache_hit":0,"new_prefill":4789,"score_linear":154908.0,"score_lmetric":51543},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":137188,"ongoing_decode_tokens":93040,"pending_prefill_tokens":2676,"num_requests":2,"active_p_offloads":0,"cached_blocks":1620,"cache_hit":0,"new_prefill":4789,"score_linear":137188.0,"score_lmetric":14930},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":255224,"ongoing_decode_tokens":60212,"pending_prefill_tokens":15812,"num_requests":4,"active_p_offloads":0,"cached_blocks":2083,"cache_hit":0,"new_prefill":4789,"score_linear":255224.0,"score_lmetric":82404},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":34445,"ongoing_decode_tokens":34445,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":2010,"cache_hit":0,"new_prefill":4789,"score_linear":34445.0,"score_lmetric":9578},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":302532,"ongoing_decode_tokens":190803,"pending_prefill_tokens":5233,"num_requests":3,"active_p_offloads":0,"cached_blocks":2564,"cache_hit":0,"new_prefill":4789,"score_linear":302532.0,"score_lmetric":30066},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":204632,"ongoing_decode_tokens":204632,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1622,"cache_hit":0,"new_prefill":4789,"score_linear":204632.0,"score_lmetric":14367},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1415,"cache_hit":2048,"new_prefill":2741,"score_linear":-2048.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":250216,"ongoing_decode_tokens":0,"pending_prefill_tokens":116584,"num_requests":3,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":0,"new_prefill":4789,"score_linear":250216.0,"score_lmetric":364119}],"chosen_score_linear":-2048.0,"chosen_score_lmetric":0,"t_first_token":348902.793134824,"t_first_token_unix":1779779767.807078,"t_done":348903.012158845,"t_done_unix":1779779768.0261023},{"request_id":"1381387:5:1404170:768","session_id":"1381387","input_length":69508,"t_proxy_recv":348899.427916766,"t_decision_unix":1779779764.4418585,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":68096,"affinity_cache_ratio":0.9796857915635611,"affinity_num_requests":2,"avg_num_requests":2.25,"fallback_score":null,"tie_break_used":false,"cache_hit":68096,"estimated_new_tokens":1412,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":154908,"ongoing_decode_tokens":21684,"pending_prefill_tokens":12392,"num_requests":3,"active_p_offloads":0,"cached_blocks":1554,"cache_hit":0,"new_prefill":69508,"score_linear":154908.0,"score_lmetric":245700},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":137188,"ongoing_decode_tokens":93040,"pending_prefill_tokens":2676,"num_requests":2,"active_p_offloads":0,"cached_blocks":1620,"cache_hit":0,"new_prefill":69508,"score_linear":137188.0,"score_lmetric":144368},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":255224,"ongoing_decode_tokens":60212,"pending_prefill_tokens":15812,"num_requests":4,"active_p_offloads":0,"cached_blocks":2083,"cache_hit":0,"new_prefill":69508,"score_linear":255224.0,"score_lmetric":341280},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":32944,"ongoing_decode_tokens":32944,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2010,"cache_hit":0,"new_prefill":69508,"score_linear":32944.0,"score_lmetric":69508},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":302532,"ongoing_decode_tokens":190803,"pending_prefill_tokens":5233,"num_requests":3,"active_p_offloads":0,"cached_blocks":2564,"cache_hit":0,"new_prefill":69508,"score_linear":302532.0,"score_lmetric":224223},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":135124,"ongoing_decode_tokens":135124,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1622,"cache_hit":68096,"new_prefill":1412,"score_linear":67028.0,"score_lmetric":2824},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1408,"cache_hit":0,"new_prefill":69508,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":250216,"ongoing_decode_tokens":0,"pending_prefill_tokens":116584,"num_requests":3,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":0,"new_prefill":69508,"score_linear":250216.0,"score_lmetric":558276}],"chosen_score_linear":67028.0,"chosen_score_lmetric":2824,"t_first_token":348900.051019599,"t_first_token_unix":1779779765.064963,"t_done":348903.404304021,"t_done_unix":1779779768.4182477},{"request_id":"1405155:1:1405155:775","session_id":"1405155","input_length":32944,"t_proxy_recv":348893.733712055,"t_decision_unix":1779779758.7476535,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.75,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":32944,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":21684,"ongoing_decode_tokens":21684,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1554,"cache_hit":0,"new_prefill":32944,"score_linear":21684.0,"score_lmetric":32944},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":137188,"ongoing_decode_tokens":0,"pending_prefill_tokens":95204,"num_requests":2,"active_p_offloads":0,"cached_blocks":1620,"cache_hit":0,"new_prefill":32944,"score_linear":137188.0,"score_lmetric":256296},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":145650,"ongoing_decode_tokens":0,"pending_prefill_tokens":26354,"num_requests":3,"active_p_offloads":0,"cached_blocks":2083,"cache_hit":0,"new_prefill":32944,"score_linear":145650.0,"score_lmetric":177894},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2010,"cache_hit":0,"new_prefill":32944,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":302125,"ongoing_decode_tokens":113401,"pending_prefill_tokens":5428,"num_requests":3,"active_p_offloads":0,"cached_blocks":2564,"cache_hit":0,"new_prefill":32944,"score_linear":302125.0,"score_lmetric":115116},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":203233,"ongoing_decode_tokens":135124,"pending_prefill_tokens":17421,"num_requests":3,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":512,"new_prefill":32432,"score_linear":202721.0,"score_lmetric":149559},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":24442,"ongoing_decode_tokens":24442,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1385,"cache_hit":0,"new_prefill":32944,"score_linear":24442.0,"score_lmetric":32944},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":124090,"ongoing_decode_tokens":0,"pending_prefill_tokens":111290,"num_requests":1,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":0,"new_prefill":32944,"score_linear":124090.0,"score_lmetric":144234}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348898.327198158,"t_first_token_unix":1779779763.341142,"t_done":348903.749763703,"t_done_unix":1779779768.7637067},{"request_id":"1407599:1:1407599:788","session_id":"1407599","input_length":1501,"t_proxy_recv":348902.25805817,"t_decision_unix":1779779767.2719996,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.5,"fallback_score":1501,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":1501,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":154908,"ongoing_decode_tokens":21684,"pending_prefill_tokens":12392,"num_requests":3,"active_p_offloads":0,"cached_blocks":1554,"cache_hit":0,"new_prefill":1501,"score_linear":154908.0,"score_lmetric":41679},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":137188,"ongoing_decode_tokens":93040,"pending_prefill_tokens":2676,"num_requests":2,"active_p_offloads":0,"cached_blocks":1620,"cache_hit":0,"new_prefill":1501,"score_linear":137188.0,"score_lmetric":8354},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":255224,"ongoing_decode_tokens":60212,"pending_prefill_tokens":15812,"num_requests":4,"active_p_offloads":0,"cached_blocks":2083,"cache_hit":0,"new_prefill":1501,"score_linear":255224.0,"score_lmetric":69252},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":32944,"ongoing_decode_tokens":32944,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2010,"cache_hit":0,"new_prefill":1501,"score_linear":32944.0,"score_lmetric":1501},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":302532,"ongoing_decode_tokens":190803,"pending_prefill_tokens":5233,"num_requests":3,"active_p_offloads":0,"cached_blocks":2564,"cache_hit":0,"new_prefill":1501,"score_linear":302532.0,"score_lmetric":20202},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":204632,"ongoing_decode_tokens":204632,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1622,"cache_hit":0,"new_prefill":1501,"score_linear":204632.0,"score_lmetric":4503},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":1531,"ongoing_decode_tokens":1531,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1412,"cache_hit":0,"new_prefill":1501,"score_linear":1531.0,"score_lmetric":1501},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":250216,"ongoing_decode_tokens":0,"pending_prefill_tokens":116584,"num_requests":3,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":0,"new_prefill":1501,"score_linear":250216.0,"score_lmetric":354255}],"chosen_score_linear":32944.0,"chosen_score_lmetric":1501,"t_first_token":348902.415127971,"t_first_token_unix":1779779767.429071,"t_done":348903.856772583,"t_done_unix":1779779768.8707163},{"request_id":"1407995:1:1407995:793","session_id":"1407995","input_length":2944,"t_proxy_recv":348903.724599796,"t_decision_unix":1779779768.7385414,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.5,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":2944,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":154908,"ongoing_decode_tokens":21684,"pending_prefill_tokens":12392,"num_requests":3,"active_p_offloads":0,"cached_blocks":1554,"cache_hit":0,"new_prefill":2944,"score_linear":154908.0,"score_lmetric":46008},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":137188,"ongoing_decode_tokens":137188,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1620,"cache_hit":0,"new_prefill":2944,"score_linear":137188.0,"score_lmetric":5888},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":255224,"ongoing_decode_tokens":60212,"pending_prefill_tokens":15812,"num_requests":4,"active_p_offloads":0,"cached_blocks":2083,"cache_hit":0,"new_prefill":2944,"score_linear":255224.0,"score_lmetric":75024},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":34445,"ongoing_decode_tokens":34445,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":2010,"cache_hit":0,"new_prefill":2944,"score_linear":34445.0,"score_lmetric":5888},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":302532,"ongoing_decode_tokens":190803,"pending_prefill_tokens":5233,"num_requests":3,"active_p_offloads":0,"cached_blocks":2564,"cache_hit":0,"new_prefill":2944,"score_linear":302532.0,"score_lmetric":24531},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":162800,"ongoing_decode_tokens":135124,"pending_prefill_tokens":5660,"num_requests":3,"active_p_offloads":0,"cached_blocks":1625,"cache_hit":0,"new_prefill":2944,"score_linear":162800.0,"score_lmetric":25812},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1420,"cache_hit":0,"new_prefill":2944,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":250216,"ongoing_decode_tokens":0,"pending_prefill_tokens":116584,"num_requests":3,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":0,"new_prefill":2944,"score_linear":250216.0,"score_lmetric":358584}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348903.929745194,"t_first_token_unix":1779779768.9436886,"t_done":348904.310427066,"t_done_unix":1779779769.3243704},{"request_id":"1397515:1:1397515:736","session_id":"1397515","input_length":93040,"t_proxy_recv":348866.983681848,"t_decision_unix":1779779731.997623,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":false,"cache_hit":512,"estimated_new_tokens":92528,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1519,"cache_hit":0,"new_prefill":93040,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1620,"cache_hit":512,"new_prefill":92528,"score_linear":-512.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":85504,"ongoing_decode_tokens":85504,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2002,"cache_hit":0,"new_prefill":93040,"score_linear":85504.0,"score_lmetric":93040},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":13134,"ongoing_decode_tokens":13134,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1940,"cache_hit":0,"new_prefill":93040,"score_linear":13134.0,"score_lmetric":93040},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":246162,"pending_prefill_tokens":201,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":93040,"score_linear":296027.0,"score_lmetric":279723},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":135124,"ongoing_decode_tokens":0,"pending_prefill_tokens":33748,"num_requests":2,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":93040,"score_linear":135124.0,"score_lmetric":253576},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":145452,"ongoing_decode_tokens":23641,"pending_prefill_tokens":59859,"num_requests":2,"active_p_offloads":0,"cached_blocks":1348,"cache_hit":0,"new_prefill":93040,"score_linear":145452.0,"score_lmetric":305798},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":101993,"ongoing_decode_tokens":101993,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1288,"cache_hit":0,"new_prefill":93040,"score_linear":101993.0,"score_lmetric":93040}],"chosen_score_linear":-512.0,"chosen_score_lmetric":0,"t_first_token":348895.922310826,"t_first_token_unix":1779779760.9362545,"t_done":348905.612468535,"t_done_unix":1779779770.6264126},{"request_id":"1373431:2:1378320:629","session_id":"1373431","input_length":127267,"t_proxy_recv":348878.230532478,"t_decision_unix":1779779743.2444742,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":126464,"affinity_cache_ratio":0.9936904303550803,"affinity_num_requests":2,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":126464,"estimated_new_tokens":803,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1530,"cache_hit":0,"new_prefill":127267,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":93040,"ongoing_decode_tokens":0,"pending_prefill_tokens":92528,"num_requests":1,"active_p_offloads":0,"cached_blocks":1620,"cache_hit":1024,"new_prefill":126243,"score_linear":92016.0,"score_lmetric":218771},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":124097,"ongoing_decode_tokens":26440,"pending_prefill_tokens":12153,"num_requests":2,"active_p_offloads":0,"cached_blocks":2036,"cache_hit":0,"new_prefill":127267,"score_linear":124097.0,"score_lmetric":278840},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76392,"ongoing_decode_tokens":13134,"pending_prefill_tokens":12058,"num_requests":2,"active_p_offloads":0,"cached_blocks":1940,"cache_hit":1024,"new_prefill":126243,"score_linear":75368.0,"score_lmetric":276602},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":169687,"ongoing_decode_tokens":119822,"pending_prefill_tokens":201,"num_requests":2,"active_p_offloads":0,"cached_blocks":2489,"cache_hit":126464,"new_prefill":803,"score_linear":43223.0,"score_lmetric":2008},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":135124,"ongoing_decode_tokens":0,"pending_prefill_tokens":33748,"num_requests":2,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":127267,"score_linear":135124.0,"score_lmetric":322030},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":145452,"ongoing_decode_tokens":23641,"pending_prefill_tokens":59859,"num_requests":2,"active_p_offloads":0,"cached_blocks":1348,"cache_hit":0,"new_prefill":127267,"score_linear":145452.0,"score_lmetric":374252},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":11414,"ongoing_decode_tokens":0,"pending_prefill_tokens":7830,"num_requests":1,"active_p_offloads":0,"cached_blocks":1327,"cache_hit":0,"new_prefill":127267,"score_linear":11414.0,"score_lmetric":135097}],"chosen_score_linear":43223.0,"chosen_score_lmetric":2008,"t_first_token":348895.728491081,"t_first_token_unix":1779779760.7424421,"t_done":348905.643190341,"t_done_unix":1779779770.657134},{"request_id":"1403188:2:1408249:796","session_id":"1403188","input_length":14728,"t_proxy_recv":348904.682879022,"t_decision_unix":1779779769.6968203,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":8704,"affinity_cache_ratio":0.5909831613253667,"affinity_num_requests":0,"avg_num_requests":2.375,"fallback_score":null,"tie_break_used":false,"cache_hit":8704,"estimated_new_tokens":6024,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":154908,"ongoing_decode_tokens":21684,"pending_prefill_tokens":12392,"num_requests":3,"active_p_offloads":0,"cached_blocks":1554,"cache_hit":0,"new_prefill":14728,"score_linear":154908.0,"score_lmetric":81360},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":137188,"ongoing_decode_tokens":137188,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1620,"cache_hit":0,"new_prefill":14728,"score_linear":137188.0,"score_lmetric":29456},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":283010,"ongoing_decode_tokens":60212,"pending_prefill_tokens":16462,"num_requests":5,"active_p_offloads":0,"cached_blocks":2083,"cache_hit":0,"new_prefill":14728,"score_linear":283010.0,"score_lmetric":155950},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2078,"cache_hit":8704,"new_prefill":6024,"score_linear":-8704.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":302532,"ongoing_decode_tokens":190803,"pending_prefill_tokens":5233,"num_requests":3,"active_p_offloads":0,"cached_blocks":2564,"cache_hit":0,"new_prefill":14728,"score_linear":302532.0,"score_lmetric":59883},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":162800,"ongoing_decode_tokens":135124,"pending_prefill_tokens":5660,"num_requests":3,"active_p_offloads":0,"cached_blocks":1625,"cache_hit":0,"new_prefill":14728,"score_linear":162800.0,"score_lmetric":61164},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1425,"cache_hit":0,"new_prefill":14728,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":250216,"ongoing_decode_tokens":0,"pending_prefill_tokens":116584,"num_requests":3,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":0,"new_prefill":14728,"score_linear":250216.0,"score_lmetric":393936}],"chosen_score_linear":-8704.0,"chosen_score_lmetric":0,"t_first_token":348905.36107854,"t_first_token_unix":1779779770.375022,"t_done":348905.651919016,"t_done_unix":1779779770.665862},{"request_id":"1408430:1:1408430:797","session_id":"1408430","input_length":7553,"t_proxy_recv":348905.256483732,"t_decision_unix":1779779770.270425,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":6,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.5,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":7553,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":154908,"ongoing_decode_tokens":21684,"pending_prefill_tokens":12392,"num_requests":3,"active_p_offloads":0,"cached_blocks":1554,"cache_hit":0,"new_prefill":7553,"score_linear":154908.0,"score_lmetric":59835},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":137188,"ongoing_decode_tokens":137188,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1620,"cache_hit":0,"new_prefill":7553,"score_linear":137188.0,"score_lmetric":15106},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":283010,"ongoing_decode_tokens":60212,"pending_prefill_tokens":16462,"num_requests":5,"active_p_offloads":0,"cached_blocks":2083,"cache_hit":0,"new_prefill":7553,"score_linear":283010.0,"score_lmetric":120075},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":14728,"ongoing_decode_tokens":0,"pending_prefill_tokens":6024,"num_requests":1,"active_p_offloads":0,"cached_blocks":2078,"cache_hit":0,"new_prefill":7553,"score_linear":14728.0,"score_lmetric":13577},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":302532,"ongoing_decode_tokens":252260,"pending_prefill_tokens":608,"num_requests":3,"active_p_offloads":0,"cached_blocks":2564,"cache_hit":0,"new_prefill":7553,"score_linear":302532.0,"score_lmetric":24483},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":162800,"ongoing_decode_tokens":135124,"pending_prefill_tokens":5660,"num_requests":3,"active_p_offloads":0,"cached_blocks":1625,"cache_hit":0,"new_prefill":7553,"score_linear":162800.0,"score_lmetric":39639},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1425,"cache_hit":0,"new_prefill":7553,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":250216,"ongoing_decode_tokens":0,"pending_prefill_tokens":116584,"num_requests":3,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":0,"new_prefill":7553,"score_linear":250216.0,"score_lmetric":372411}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348905.774921838,"t_first_token_unix":1779779770.7888653,"t_done":348905.969943047,"t_done_unix":1779779770.9838862},{"request_id":"1268861:8:1403753:765","session_id":"1268861","input_length":44148,"t_proxy_recv":348888.867996515,"t_decision_unix":1779779753.881938,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":41472,"affinity_cache_ratio":0.939385702636586,"affinity_num_requests":1,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":41472,"estimated_new_tokens":2676,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1554,"cache_hit":0,"new_prefill":44148,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":93040,"ongoing_decode_tokens":0,"pending_prefill_tokens":92528,"num_requests":1,"active_p_offloads":0,"cached_blocks":1620,"cache_hit":41472,"new_prefill":2676,"score_linear":51568.0,"score_lmetric":95204},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2083,"cache_hit":0,"new_prefill":44148,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2006,"cache_hit":0,"new_prefill":44148,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":302125,"ongoing_decode_tokens":113401,"pending_prefill_tokens":5428,"num_requests":3,"active_p_offloads":0,"cached_blocks":2564,"cache_hit":0,"new_prefill":44148,"score_linear":302125.0,"score_lmetric":148728},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":203233,"ongoing_decode_tokens":101803,"pending_prefill_tokens":22582,"num_requests":3,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":44148,"score_linear":203233.0,"score_lmetric":200190},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":85659,"ongoing_decode_tokens":85659,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1380,"cache_hit":0,"new_prefill":44148,"score_linear":85659.0,"score_lmetric":88296},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":75540,"ongoing_decode_tokens":75540,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1342,"cache_hit":12288,"new_prefill":31860,"score_linear":63252.0,"score_lmetric":63720}],"chosen_score_linear":51568.0,"chosen_score_lmetric":95204,"t_first_token":348903.050966088,"t_first_token_unix":1779779768.0649116,"t_done":348906.158033849,"t_done_unix":1779779771.1719773},{"request_id":"1408779:1:1408779:802","session_id":"1408779","input_length":1801,"t_proxy_recv":348906.362660684,"t_decision_unix":1779779771.3766017,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.375,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":1801,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":154908,"ongoing_decode_tokens":21684,"pending_prefill_tokens":12392,"num_requests":3,"active_p_offloads":0,"cached_blocks":1554,"cache_hit":0,"new_prefill":1801,"score_linear":154908.0,"score_lmetric":42579},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1806,"cache_hit":0,"new_prefill":1801,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":283010,"ongoing_decode_tokens":60212,"pending_prefill_tokens":16462,"num_requests":5,"active_p_offloads":0,"cached_blocks":2083,"cache_hit":0,"new_prefill":1801,"score_linear":283010.0,"score_lmetric":91315},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":6377,"ongoing_decode_tokens":6377,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2089,"cache_hit":0,"new_prefill":1801,"score_linear":6377.0,"score_lmetric":1801},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":303466,"ongoing_decode_tokens":124993,"pending_prefill_tokens":1833,"num_requests":3,"active_p_offloads":0,"cached_blocks":2565,"cache_hit":0,"new_prefill":1801,"score_linear":303466.0,"score_lmetric":10902},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":162800,"ongoing_decode_tokens":162800,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1625,"cache_hit":0,"new_prefill":1801,"score_linear":162800.0,"score_lmetric":5403},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":11070,"ongoing_decode_tokens":0,"pending_prefill_tokens":11070,"num_requests":1,"active_p_offloads":0,"cached_blocks":1439,"cache_hit":0,"new_prefill":1801,"score_linear":11070.0,"score_lmetric":12871},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":250216,"ongoing_decode_tokens":0,"pending_prefill_tokens":116584,"num_requests":3,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":0,"new_prefill":1801,"score_linear":250216.0,"score_lmetric":355155}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348906.490481324,"t_first_token_unix":1779779771.5044248,"t_done":348906.721046222,"t_done_unix":1779779771.73499},{"request_id":"1408889:1:1408889:804","session_id":"1408889","input_length":3230,"t_proxy_recv":348906.726159112,"t_decision_unix":1779779771.7401006,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":1,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.5,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":3230,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":154908,"ongoing_decode_tokens":21684,"pending_prefill_tokens":12392,"num_requests":3,"active_p_offloads":0,"cached_blocks":1554,"cache_hit":0,"new_prefill":3230,"score_linear":154908.0,"score_lmetric":46866},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1809,"cache_hit":0,"new_prefill":3230,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":283010,"ongoing_decode_tokens":60212,"pending_prefill_tokens":16462,"num_requests":5,"active_p_offloads":0,"cached_blocks":2083,"cache_hit":0,"new_prefill":3230,"score_linear":283010.0,"score_lmetric":98460},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":6377,"ongoing_decode_tokens":6377,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2089,"cache_hit":0,"new_prefill":3230,"score_linear":6377.0,"score_lmetric":3230},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":303466,"ongoing_decode_tokens":124993,"pending_prefill_tokens":1833,"num_requests":3,"active_p_offloads":0,"cached_blocks":2565,"cache_hit":0,"new_prefill":3230,"score_linear":303466.0,"score_lmetric":15189},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":162800,"ongoing_decode_tokens":162800,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1625,"cache_hit":0,"new_prefill":3230,"score_linear":162800.0,"score_lmetric":9690},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":11070,"ongoing_decode_tokens":0,"pending_prefill_tokens":11070,"num_requests":1,"active_p_offloads":0,"cached_blocks":1439,"cache_hit":0,"new_prefill":3230,"score_linear":11070.0,"score_lmetric":14300},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":323608,"ongoing_decode_tokens":0,"pending_prefill_tokens":127000,"num_requests":4,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":0,"new_prefill":3230,"score_linear":323608.0,"score_lmetric":520920}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348906.921393119,"t_first_token_unix":1779779771.9353364,"t_done":348907.047643116,"t_done_unix":1779779772.0615869},{"request_id":"1408578:1:1408578:798","session_id":"1408578","input_length":6377,"t_proxy_recv":348905.684772941,"t_decision_unix":1779779770.6987147,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":3,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":2.375,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":6377,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":154908,"ongoing_decode_tokens":21684,"pending_prefill_tokens":12392,"num_requests":3,"active_p_offloads":0,"cached_blocks":1554,"cache_hit":0,"new_prefill":6377,"score_linear":154908.0,"score_lmetric":56307},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":44148,"ongoing_decode_tokens":44148,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1801,"cache_hit":0,"new_prefill":6377,"score_linear":44148.0,"score_lmetric":6377},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":283010,"ongoing_decode_tokens":60212,"pending_prefill_tokens":16462,"num_requests":5,"active_p_offloads":0,"cached_blocks":2083,"cache_hit":0,"new_prefill":6377,"score_linear":283010.0,"score_lmetric":114195},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2089,"cache_hit":0,"new_prefill":6377,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":303466,"ongoing_decode_tokens":124993,"pending_prefill_tokens":1833,"num_requests":3,"active_p_offloads":0,"cached_blocks":2565,"cache_hit":0,"new_prefill":6377,"score_linear":303466.0,"score_lmetric":24630},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":162800,"ongoing_decode_tokens":135124,"pending_prefill_tokens":5660,"num_requests":3,"active_p_offloads":0,"cached_blocks":1625,"cache_hit":0,"new_prefill":6377,"score_linear":162800.0,"score_lmetric":36111},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":7553,"ongoing_decode_tokens":0,"pending_prefill_tokens":7553,"num_requests":1,"active_p_offloads":0,"cached_blocks":1425,"cache_hit":0,"new_prefill":6377,"score_linear":7553.0,"score_lmetric":13930},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":250216,"ongoing_decode_tokens":0,"pending_prefill_tokens":116584,"num_requests":3,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":0,"new_prefill":6377,"score_linear":250216.0,"score_lmetric":368883}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348906.105655121,"t_first_token_unix":1779779771.1195986,"t_done":348907.194877818,"t_done_unix":1779779772.2088213},{"request_id":"1362265:7:1408772:800","session_id":"1362265","input_length":11070,"t_proxy_recv":348906.339851994,"t_decision_unix":1779779771.3537934,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":2,"chosen_idx":6,"affinity_cache_hit":10752,"affinity_cache_ratio":0.9712737127371274,"affinity_num_requests":5,"avg_num_requests":2.25,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":11070,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":154908,"ongoing_decode_tokens":21684,"pending_prefill_tokens":12392,"num_requests":3,"active_p_offloads":0,"cached_blocks":1554,"cache_hit":0,"new_prefill":11070,"score_linear":154908.0,"score_lmetric":70386},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1806,"cache_hit":0,"new_prefill":11070,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":283010,"ongoing_decode_tokens":60212,"pending_prefill_tokens":16462,"num_requests":5,"active_p_offloads":0,"cached_blocks":2083,"cache_hit":10752,"new_prefill":318,"score_linear":272258.0,"score_lmetric":83900},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":6377,"ongoing_decode_tokens":6377,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2089,"cache_hit":0,"new_prefill":11070,"score_linear":6377.0,"score_lmetric":11070},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":303466,"ongoing_decode_tokens":124993,"pending_prefill_tokens":1833,"num_requests":3,"active_p_offloads":0,"cached_blocks":2565,"cache_hit":0,"new_prefill":11070,"score_linear":303466.0,"score_lmetric":38709},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":162800,"ongoing_decode_tokens":162800,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1625,"cache_hit":0,"new_prefill":11070,"score_linear":162800.0,"score_lmetric":33210},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1439,"cache_hit":0,"new_prefill":11070,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":250216,"ongoing_decode_tokens":0,"pending_prefill_tokens":116584,"num_requests":3,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":0,"new_prefill":11070,"score_linear":250216.0,"score_lmetric":382962}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348907.219931826,"t_first_token_unix":1779779772.2338755,"t_done":348907.709396767,"t_done_unix":1779779772.7233403},{"request_id":"1408779:2:1410032:807","session_id":"1408779","input_length":5001,"t_proxy_recv":348910.878420567,"t_decision_unix":1779779775.8923616,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":1536,"affinity_cache_ratio":0.30713857228554287,"affinity_num_requests":0,"avg_num_requests":2.25,"fallback_score":0,"tie_break_used":false,"cache_hit":1536,"estimated_new_tokens":3465,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":154908,"ongoing_decode_tokens":21684,"pending_prefill_tokens":12392,"num_requests":3,"active_p_offloads":0,"cached_blocks":1554,"cache_hit":0,"new_prefill":5001,"score_linear":154908.0,"score_lmetric":52179},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1815,"cache_hit":1536,"new_prefill":3465,"score_linear":-1536.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":283010,"ongoing_decode_tokens":60212,"pending_prefill_tokens":16462,"num_requests":4,"active_p_offloads":0,"cached_blocks":2083,"cache_hit":0,"new_prefill":5001,"score_linear":283010.0,"score_lmetric":85852},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2101,"cache_hit":0,"new_prefill":5001,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":303466,"ongoing_decode_tokens":124993,"pending_prefill_tokens":1833,"num_requests":3,"active_p_offloads":0,"cached_blocks":2565,"cache_hit":0,"new_prefill":5001,"score_linear":303466.0,"score_lmetric":20502},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":234370,"ongoing_decode_tokens":234370,"pending_prefill_tokens":0,"num_requests":4,"active_p_offloads":0,"cached_blocks":1625,"cache_hit":0,"new_prefill":5001,"score_linear":234370.0,"score_lmetric":20004},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1460,"cache_hit":0,"new_prefill":5001,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":323608,"ongoing_decode_tokens":0,"pending_prefill_tokens":127000,"num_requests":4,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":0,"new_prefill":5001,"score_linear":323608.0,"score_lmetric":528004}],"chosen_score_linear":-1536.0,"chosen_score_lmetric":0,"t_first_token":348911.107280579,"t_first_token_unix":1779779776.1212246,"t_done":348911.25670017,"t_done_unix":1779779776.2706435},{"request_id":"1406966:3:1410097:808","session_id":"1406966","input_length":7893,"t_proxy_recv":348911.116649129,"t_decision_unix":1779779776.1305907,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":4608,"affinity_cache_ratio":0.5838084378563284,"affinity_num_requests":0,"avg_num_requests":2.375,"fallback_score":null,"tie_break_used":false,"cache_hit":4608,"estimated_new_tokens":3285,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":154908,"ongoing_decode_tokens":21684,"pending_prefill_tokens":12392,"num_requests":3,"active_p_offloads":0,"cached_blocks":1554,"cache_hit":0,"new_prefill":7893,"score_linear":154908.0,"score_lmetric":60855},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":5001,"ongoing_decode_tokens":5001,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1815,"cache_hit":0,"new_prefill":7893,"score_linear":5001.0,"score_lmetric":7893},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":283010,"ongoing_decode_tokens":60212,"pending_prefill_tokens":16462,"num_requests":4,"active_p_offloads":0,"cached_blocks":2083,"cache_hit":0,"new_prefill":7893,"score_linear":283010.0,"score_lmetric":97420},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2101,"cache_hit":0,"new_prefill":7893,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":303466,"ongoing_decode_tokens":124993,"pending_prefill_tokens":1833,"num_requests":3,"active_p_offloads":0,"cached_blocks":2565,"cache_hit":0,"new_prefill":7893,"score_linear":303466.0,"score_lmetric":29178},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":234370,"ongoing_decode_tokens":234370,"pending_prefill_tokens":0,"num_requests":4,"active_p_offloads":0,"cached_blocks":1625,"cache_hit":0,"new_prefill":7893,"score_linear":234370.0,"score_lmetric":31572},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1460,"cache_hit":4608,"new_prefill":3285,"score_linear":-4608.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":323608,"ongoing_decode_tokens":0,"pending_prefill_tokens":127000,"num_requests":4,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":0,"new_prefill":7893,"score_linear":323608.0,"score_lmetric":539572}],"chosen_score_linear":-4608.0,"chosen_score_lmetric":0,"t_first_token":348911.39584079,"t_first_token_unix":1779779776.4097846,"t_done":348911.593441834,"t_done_unix":1779779776.6073854},{"request_id":"1286804:5:1407989:792","session_id":"1286804","input_length":27676,"t_proxy_recv":348903.701021487,"t_decision_unix":1779779768.7149627,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":22016,"affinity_cache_ratio":0.795490677843619,"affinity_num_requests":2,"avg_num_requests":2.375,"fallback_score":null,"tie_break_used":false,"cache_hit":22016,"estimated_new_tokens":5660,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":154908,"ongoing_decode_tokens":21684,"pending_prefill_tokens":12392,"num_requests":3,"active_p_offloads":0,"cached_blocks":1554,"cache_hit":0,"new_prefill":27676,"score_linear":154908.0,"score_lmetric":120204},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":137188,"ongoing_decode_tokens":137188,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1620,"cache_hit":0,"new_prefill":27676,"score_linear":137188.0,"score_lmetric":55352},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":255224,"ongoing_decode_tokens":60212,"pending_prefill_tokens":15812,"num_requests":4,"active_p_offloads":0,"cached_blocks":2083,"cache_hit":0,"new_prefill":27676,"score_linear":255224.0,"score_lmetric":173952},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":34445,"ongoing_decode_tokens":34445,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":2010,"cache_hit":0,"new_prefill":27676,"score_linear":34445.0,"score_lmetric":55352},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":302532,"ongoing_decode_tokens":190803,"pending_prefill_tokens":5233,"num_requests":3,"active_p_offloads":0,"cached_blocks":2564,"cache_hit":0,"new_prefill":27676,"score_linear":302532.0,"score_lmetric":98727},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":135124,"ongoing_decode_tokens":135124,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1625,"cache_hit":22016,"new_prefill":5660,"score_linear":113108.0,"score_lmetric":11320},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1420,"cache_hit":0,"new_prefill":27676,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":250216,"ongoing_decode_tokens":0,"pending_prefill_tokens":116584,"num_requests":3,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":0,"new_prefill":27676,"score_linear":250216.0,"score_lmetric":432780}],"chosen_score_linear":113108.0,"chosen_score_lmetric":11320,"t_first_token":348906.197762019,"t_first_token_unix":1779779771.2117054,"t_done":348911.622742103,"t_done_unix":1779779776.6366851},{"request_id":"1340278:3:1385417:683","session_id":"1340278","input_length":50272,"t_proxy_recv":348897.570591984,"t_decision_unix":1779779762.5845335,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":49664,"affinity_cache_ratio":0.9879057924888606,"affinity_num_requests":2,"avg_num_requests":2.25,"fallback_score":null,"tie_break_used":false,"cache_hit":49664,"estimated_new_tokens":608,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":154908,"ongoing_decode_tokens":21684,"pending_prefill_tokens":12392,"num_requests":3,"active_p_offloads":0,"cached_blocks":1554,"cache_hit":0,"new_prefill":50272,"score_linear":154908.0,"score_lmetric":187992},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":137188,"ongoing_decode_tokens":93040,"pending_prefill_tokens":2676,"num_requests":2,"active_p_offloads":0,"cached_blocks":1620,"cache_hit":0,"new_prefill":50272,"score_linear":137188.0,"score_lmetric":105896},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":255224,"ongoing_decode_tokens":60212,"pending_prefill_tokens":15812,"num_requests":4,"active_p_offloads":0,"cached_blocks":2083,"cache_hit":0,"new_prefill":50272,"score_linear":255224.0,"score_lmetric":264336},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":32944,"ongoing_decode_tokens":0,"pending_prefill_tokens":32944,"num_requests":1,"active_p_offloads":0,"cached_blocks":2010,"cache_hit":0,"new_prefill":50272,"score_linear":32944.0,"score_lmetric":83216},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":252260,"ongoing_decode_tokens":190803,"pending_prefill_tokens":4625,"num_requests":2,"active_p_offloads":0,"cached_blocks":2564,"cache_hit":49664,"new_prefill":608,"score_linear":202596.0,"score_lmetric":10466},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":203233,"ongoing_decode_tokens":203233,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":50272,"score_linear":203233.0,"score_lmetric":150816},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1408,"cache_hit":0,"new_prefill":50272,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":250216,"ongoing_decode_tokens":0,"pending_prefill_tokens":116584,"num_requests":3,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":0,"new_prefill":50272,"score_linear":250216.0,"score_lmetric":500568}],"chosen_score_linear":202596.0,"chosen_score_lmetric":10466,"t_first_token":348911.614180408,"t_first_token_unix":1779779776.6281242,"t_done":348911.676924581,"t_done_unix":1779779776.690868},{"request_id":"1381387:6:1409195:806","session_id":"1381387","input_length":71570,"t_proxy_recv":348907.997775801,"t_decision_unix":1779779773.0117173,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":69632,"affinity_cache_ratio":0.9729216152019002,"affinity_num_requests":3,"avg_num_requests":2.125,"fallback_score":null,"tie_break_used":false,"cache_hit":69632,"estimated_new_tokens":1938,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":154908,"ongoing_decode_tokens":21684,"pending_prefill_tokens":12392,"num_requests":3,"active_p_offloads":0,"cached_blocks":1554,"cache_hit":0,"new_prefill":71570,"score_linear":154908.0,"score_lmetric":251886},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1815,"cache_hit":0,"new_prefill":71570,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":283010,"ongoing_decode_tokens":60212,"pending_prefill_tokens":16462,"num_requests":4,"active_p_offloads":0,"cached_blocks":2083,"cache_hit":0,"new_prefill":71570,"score_linear":283010.0,"score_lmetric":352128},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2101,"cache_hit":512,"new_prefill":71058,"score_linear":-512.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":303466,"ongoing_decode_tokens":124993,"pending_prefill_tokens":1833,"num_requests":3,"active_p_offloads":0,"cached_blocks":2565,"cache_hit":0,"new_prefill":71570,"score_linear":303466.0,"score_lmetric":220209},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":162800,"ongoing_decode_tokens":162800,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1625,"cache_hit":69632,"new_prefill":1938,"score_linear":93168.0,"score_lmetric":5814},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1460,"cache_hit":0,"new_prefill":71570,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":323608,"ongoing_decode_tokens":0,"pending_prefill_tokens":127000,"num_requests":4,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":0,"new_prefill":71570,"score_linear":323608.0,"score_lmetric":794280}],"chosen_score_linear":93168.0,"chosen_score_lmetric":5814,"t_first_token":348909.017063998,"t_first_token_unix":1779779774.0310075,"t_done":348911.868426288,"t_done_unix":1779779776.88237},{"request_id":"1407494:2:1410400:809","session_id":"1407494","input_length":4728,"t_proxy_recv":348911.975444014,"t_decision_unix":1779779776.9893851,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":1536,"affinity_cache_ratio":0.3248730964467005,"affinity_num_requests":0,"avg_num_requests":1.875,"fallback_score":0,"tie_break_used":false,"cache_hit":1536,"estimated_new_tokens":3192,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":154908,"ongoing_decode_tokens":21684,"pending_prefill_tokens":12392,"num_requests":3,"active_p_offloads":0,"cached_blocks":1554,"cache_hit":0,"new_prefill":4728,"score_linear":154908.0,"score_lmetric":51360},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1821,"cache_hit":0,"new_prefill":4728,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":283010,"ongoing_decode_tokens":60212,"pending_prefill_tokens":16462,"num_requests":4,"active_p_offloads":0,"cached_blocks":2083,"cache_hit":0,"new_prefill":4728,"score_linear":283010.0,"score_lmetric":84760},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2101,"cache_hit":0,"new_prefill":4728,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":253194,"ongoing_decode_tokens":124993,"pending_prefill_tokens":1225,"num_requests":2,"active_p_offloads":0,"cached_blocks":2566,"cache_hit":0,"new_prefill":4728,"score_linear":253194.0,"score_lmetric":11906},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":135124,"ongoing_decode_tokens":135124,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1640,"cache_hit":0,"new_prefill":4728,"score_linear":135124.0,"score_lmetric":9456},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1466,"cache_hit":1536,"new_prefill":3192,"score_linear":-1536.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":323608,"ongoing_decode_tokens":0,"pending_prefill_tokens":127000,"num_requests":4,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":0,"new_prefill":4728,"score_linear":323608.0,"score_lmetric":526912}],"chosen_score_linear":-1536.0,"chosen_score_lmetric":0,"t_first_token":348912.200403516,"t_first_token_unix":1779779777.2143471,"t_done":348912.336804968,"t_done_unix":1779779777.350748},{"request_id":"1408889:2:1410453:810","session_id":"1408889","input_length":4825,"t_proxy_recv":348912.113574363,"t_decision_unix":1779779777.1275158,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":3072,"affinity_cache_ratio":0.6366839378238341,"affinity_num_requests":0,"avg_num_requests":2.0,"fallback_score":null,"tie_break_used":false,"cache_hit":3072,"estimated_new_tokens":1753,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":154908,"ongoing_decode_tokens":21684,"pending_prefill_tokens":12392,"num_requests":3,"active_p_offloads":0,"cached_blocks":1554,"cache_hit":0,"new_prefill":4825,"score_linear":154908.0,"score_lmetric":51651},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1821,"cache_hit":3072,"new_prefill":1753,"score_linear":-3072.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":283010,"ongoing_decode_tokens":60212,"pending_prefill_tokens":16462,"num_requests":4,"active_p_offloads":0,"cached_blocks":2083,"cache_hit":0,"new_prefill":4825,"score_linear":283010.0,"score_lmetric":85148},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2101,"cache_hit":0,"new_prefill":4825,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":253194,"ongoing_decode_tokens":124993,"pending_prefill_tokens":1225,"num_requests":2,"active_p_offloads":0,"cached_blocks":2566,"cache_hit":0,"new_prefill":4825,"score_linear":253194.0,"score_lmetric":12100},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":135124,"ongoing_decode_tokens":135124,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1640,"cache_hit":0,"new_prefill":4825,"score_linear":135124.0,"score_lmetric":9650},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":4728,"ongoing_decode_tokens":0,"pending_prefill_tokens":3192,"num_requests":1,"active_p_offloads":0,"cached_blocks":1466,"cache_hit":0,"new_prefill":4825,"score_linear":4728.0,"score_lmetric":8017},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":323608,"ongoing_decode_tokens":0,"pending_prefill_tokens":127000,"num_requests":4,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":0,"new_prefill":4825,"score_linear":323608.0,"score_lmetric":527300}],"chosen_score_linear":-3072.0,"chosen_score_lmetric":0,"t_first_token":348912.307924973,"t_first_token_unix":1779779777.3218682,"t_done":348912.594507003,"t_done_unix":1779779777.6084504},{"request_id":"1408779:3:1411249:815","session_id":"1408779","input_length":7110,"t_proxy_recv":348914.891121885,"t_decision_unix":1779779779.905063,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":4608,"affinity_cache_ratio":0.6481012658227848,"affinity_num_requests":0,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"cache_hit":4608,"estimated_new_tokens":2502,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":154908,"ongoing_decode_tokens":21684,"pending_prefill_tokens":12392,"num_requests":3,"active_p_offloads":0,"cached_blocks":1554,"cache_hit":0,"new_prefill":7110,"score_linear":154908.0,"score_lmetric":58506},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1824,"cache_hit":4608,"new_prefill":2502,"score_linear":-4608.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":283010,"ongoing_decode_tokens":60212,"pending_prefill_tokens":16462,"num_requests":4,"active_p_offloads":0,"cached_blocks":2083,"cache_hit":0,"new_prefill":7110,"score_linear":283010.0,"score_lmetric":94288},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2101,"cache_hit":0,"new_prefill":7110,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":253194,"ongoing_decode_tokens":124993,"pending_prefill_tokens":1225,"num_requests":2,"active_p_offloads":0,"cached_blocks":2566,"cache_hit":0,"new_prefill":7110,"score_linear":253194.0,"score_lmetric":16670},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":135124,"ongoing_decode_tokens":135124,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1640,"cache_hit":0,"new_prefill":7110,"score_linear":135124.0,"score_lmetric":14220},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1472,"cache_hit":0,"new_prefill":7110,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":323608,"ongoing_decode_tokens":0,"pending_prefill_tokens":127000,"num_requests":4,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":0,"new_prefill":7110,"score_linear":323608.0,"score_lmetric":536440}],"chosen_score_linear":-4608.0,"chosen_score_lmetric":0,"t_first_token":348915.090206336,"t_first_token_unix":1779779780.1041503,"t_done":348915.24701461,"t_done_unix":1779779780.2609582},{"request_id":"1407494:3:1413526:825","session_id":"1407494","input_length":9801,"t_proxy_recv":348922.92571229,"t_decision_unix":1779779787.9396534,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":4608,"affinity_cache_ratio":0.4701561065197429,"affinity_num_requests":0,"avg_num_requests":2.25,"fallback_score":0,"tie_break_used":false,"cache_hit":4608,"estimated_new_tokens":5193,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":154908,"ongoing_decode_tokens":21684,"pending_prefill_tokens":12392,"num_requests":3,"active_p_offloads":0,"cached_blocks":1554,"cache_hit":0,"new_prefill":9801,"score_linear":154908.0,"score_lmetric":66579},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1828,"cache_hit":0,"new_prefill":9801,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":283010,"ongoing_decode_tokens":145650,"pending_prefill_tokens":12432,"num_requests":4,"active_p_offloads":0,"cached_blocks":2083,"cache_hit":0,"new_prefill":9801,"score_linear":283010.0,"score_lmetric":88932},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":107292,"ongoing_decode_tokens":0,"pending_prefill_tokens":14108,"num_requests":1,"active_p_offloads":0,"cached_blocks":2101,"cache_hit":0,"new_prefill":9801,"score_linear":107292.0,"score_lmetric":23909},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":253194,"ongoing_decode_tokens":124993,"pending_prefill_tokens":1225,"num_requests":2,"active_p_offloads":0,"cached_blocks":2566,"cache_hit":0,"new_prefill":9801,"score_linear":253194.0,"score_lmetric":22052},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":248005,"ongoing_decode_tokens":207732,"pending_prefill_tokens":1873,"num_requests":4,"active_p_offloads":0,"cached_blocks":1640,"cache_hit":0,"new_prefill":9801,"score_linear":248005.0,"score_lmetric":46696},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1472,"cache_hit":4608,"new_prefill":5193,"score_linear":-4608.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":323608,"ongoing_decode_tokens":0,"pending_prefill_tokens":127000,"num_requests":4,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":0,"new_prefill":9801,"score_linear":323608.0,"score_lmetric":547204}],"chosen_score_linear":-4608.0,"chosen_score_lmetric":0,"t_first_token":348923.398585632,"t_first_token_unix":1779779788.4125297,"t_done":348923.728803318,"t_done_unix":1779779788.7427464},{"request_id":"1407995:2:1413725:826","session_id":"1407995","input_length":22392,"t_proxy_recv":348923.552807923,"t_decision_unix":1779779788.5667493,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":6,"chosen_idx":1,"affinity_cache_hit":2560,"affinity_cache_ratio":0.1143265451947124,"affinity_num_requests":1,"avg_num_requests":2.375,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":22392,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":154908,"ongoing_decode_tokens":21684,"pending_prefill_tokens":12392,"num_requests":3,"active_p_offloads":0,"cached_blocks":1554,"cache_hit":0,"new_prefill":22392,"score_linear":154908.0,"score_lmetric":104352},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1828,"cache_hit":0,"new_prefill":22392,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":283010,"ongoing_decode_tokens":145650,"pending_prefill_tokens":12432,"num_requests":4,"active_p_offloads":0,"cached_blocks":2083,"cache_hit":0,"new_prefill":22392,"score_linear":283010.0,"score_lmetric":139296},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":107292,"ongoing_decode_tokens":0,"pending_prefill_tokens":14108,"num_requests":1,"active_p_offloads":0,"cached_blocks":2101,"cache_hit":0,"new_prefill":22392,"score_linear":107292.0,"score_lmetric":36500},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":253194,"ongoing_decode_tokens":124993,"pending_prefill_tokens":1225,"num_requests":2,"active_p_offloads":0,"cached_blocks":2566,"cache_hit":0,"new_prefill":22392,"score_linear":253194.0,"score_lmetric":47234},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":248005,"ongoing_decode_tokens":207732,"pending_prefill_tokens":1873,"num_requests":4,"active_p_offloads":0,"cached_blocks":1640,"cache_hit":0,"new_prefill":22392,"score_linear":248005.0,"score_lmetric":97060},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":9801,"ongoing_decode_tokens":9801,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1472,"cache_hit":2560,"new_prefill":19832,"score_linear":7241.0,"score_lmetric":19832},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":323608,"ongoing_decode_tokens":0,"pending_prefill_tokens":127000,"num_requests":4,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":0,"new_prefill":22392,"score_linear":323608.0,"score_lmetric":597568}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348926.001963933,"t_first_token_unix":1779779791.015907,"t_done":348926.440591863,"t_done_unix":1779779791.454535},{"request_id":"1408889:3:1414177:827","session_id":"1408889","input_length":10633,"t_proxy_recv":348925.0574538,"t_decision_unix":1779779790.0713954,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":1,"chosen_idx":6,"affinity_cache_hit":4608,"affinity_cache_ratio":0.43336781717295214,"affinity_num_requests":1,"avg_num_requests":2.375,"fallback_score":0,"tie_break_used":false,"cache_hit":0,"estimated_new_tokens":10633,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":154908,"ongoing_decode_tokens":21684,"pending_prefill_tokens":12392,"num_requests":3,"active_p_offloads":0,"cached_blocks":1554,"cache_hit":0,"new_prefill":10633,"score_linear":154908.0,"score_lmetric":69075},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":22392,"ongoing_decode_tokens":0,"pending_prefill_tokens":22392,"num_requests":1,"active_p_offloads":0,"cached_blocks":1828,"cache_hit":4608,"new_prefill":6025,"score_linear":17784.0,"score_lmetric":28417},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":283010,"ongoing_decode_tokens":145650,"pending_prefill_tokens":12432,"num_requests":4,"active_p_offloads":0,"cached_blocks":2083,"cache_hit":0,"new_prefill":10633,"score_linear":283010.0,"score_lmetric":92260},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":107292,"ongoing_decode_tokens":0,"pending_prefill_tokens":14108,"num_requests":1,"active_p_offloads":0,"cached_blocks":2101,"cache_hit":0,"new_prefill":10633,"score_linear":107292.0,"score_lmetric":24741},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":253194,"ongoing_decode_tokens":124993,"pending_prefill_tokens":1225,"num_requests":2,"active_p_offloads":0,"cached_blocks":2566,"cache_hit":0,"new_prefill":10633,"score_linear":253194.0,"score_lmetric":23716},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":248005,"ongoing_decode_tokens":207732,"pending_prefill_tokens":1873,"num_requests":4,"active_p_offloads":0,"cached_blocks":1640,"cache_hit":0,"new_prefill":10633,"score_linear":248005.0,"score_lmetric":50024},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1482,"cache_hit":0,"new_prefill":10633,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":323608,"ongoing_decode_tokens":0,"pending_prefill_tokens":127000,"num_requests":4,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":0,"new_prefill":10633,"score_linear":323608.0,"score_lmetric":550532}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348925.888818692,"t_first_token_unix":1779779790.9027624,"t_done":348926.898073051,"t_done_unix":1779779791.9120162},{"request_id":"1398594:2:1404702:771","session_id":"1398594","input_length":21009,"t_proxy_recv":348892.126249222,"t_decision_unix":1779779757.1401906,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":7,"chosen_idx":2,"affinity_cache_hit":7680,"affinity_cache_ratio":0.3655576181636441,"affinity_num_requests":1,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":21009,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":21684,"ongoing_decode_tokens":0,"pending_prefill_tokens":21684,"num_requests":1,"active_p_offloads":0,"cached_blocks":1554,"cache_hit":0,"new_prefill":21009,"score_linear":21684.0,"score_lmetric":42693},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":137188,"ongoing_decode_tokens":0,"pending_prefill_tokens":95204,"num_requests":2,"active_p_offloads":0,"cached_blocks":1620,"cache_hit":0,"new_prefill":21009,"score_linear":137188.0,"score_lmetric":232426},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2083,"cache_hit":0,"new_prefill":21009,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2010,"cache_hit":0,"new_prefill":21009,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":302125,"ongoing_decode_tokens":113401,"pending_prefill_tokens":5428,"num_requests":3,"active_p_offloads":0,"cached_blocks":2564,"cache_hit":0,"new_prefill":21009,"score_linear":302125.0,"score_lmetric":79311},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":203233,"ongoing_decode_tokens":135124,"pending_prefill_tokens":17421,"num_requests":3,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":21009,"score_linear":203233.0,"score_lmetric":115290},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":24442,"ongoing_decode_tokens":24442,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1385,"cache_hit":0,"new_prefill":21009,"score_linear":24442.0,"score_lmetric":21009},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":124090,"ongoing_decode_tokens":0,"pending_prefill_tokens":111290,"num_requests":1,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":7680,"new_prefill":13329,"score_linear":116410.0,"score_lmetric":124619}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348895.241426023,"t_first_token_unix":1779779760.2553694,"t_done":348928.784375937,"t_done_unix":1779779793.798319},{"request_id":"1373431:3:1383388:667","session_id":"1373431","input_length":128201,"t_proxy_recv":348905.671762762,"t_decision_unix":1779779770.6857035,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":126976,"affinity_cache_ratio":0.9904446923190927,"affinity_num_requests":2,"avg_num_requests":2.25,"fallback_score":null,"tie_break_used":false,"cache_hit":126976,"estimated_new_tokens":1225,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":154908,"ongoing_decode_tokens":21684,"pending_prefill_tokens":12392,"num_requests":3,"active_p_offloads":0,"cached_blocks":1554,"cache_hit":0,"new_prefill":128201,"score_linear":154908.0,"score_lmetric":421779},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":44148,"ongoing_decode_tokens":44148,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1801,"cache_hit":1024,"new_prefill":127177,"score_linear":43124.0,"score_lmetric":127177},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":283010,"ongoing_decode_tokens":60212,"pending_prefill_tokens":16462,"num_requests":5,"active_p_offloads":0,"cached_blocks":2083,"cache_hit":0,"new_prefill":128201,"score_linear":283010.0,"score_lmetric":723315},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2089,"cache_hit":1024,"new_prefill":127177,"score_linear":-1024.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":175265,"ongoing_decode_tokens":124993,"pending_prefill_tokens":608,"num_requests":2,"active_p_offloads":0,"cached_blocks":2565,"cache_hit":126976,"new_prefill":1225,"score_linear":48289.0,"score_lmetric":3666},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":162800,"ongoing_decode_tokens":135124,"pending_prefill_tokens":5660,"num_requests":3,"active_p_offloads":0,"cached_blocks":1625,"cache_hit":0,"new_prefill":128201,"score_linear":162800.0,"score_lmetric":401583},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":7553,"ongoing_decode_tokens":0,"pending_prefill_tokens":7553,"num_requests":1,"active_p_offloads":0,"cached_blocks":1425,"cache_hit":0,"new_prefill":128201,"score_linear":7553.0,"score_lmetric":135754},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":250216,"ongoing_decode_tokens":0,"pending_prefill_tokens":116584,"num_requests":3,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":0,"new_prefill":128201,"score_linear":250216.0,"score_lmetric":734355}],"chosen_score_linear":48289.0,"chosen_score_lmetric":3666,"t_first_token":348927.872403261,"t_first_token_unix":1779779792.886347,"t_done":348929.671012575,"t_done_unix":1779779794.6849563},{"request_id":"1382089:2:1404847:773","session_id":"1382089","input_length":39203,"t_proxy_recv":348892.632803133,"t_decision_unix":1779779757.6467443,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":37888,"affinity_cache_ratio":0.9664566487258628,"affinity_num_requests":1,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":37888,"estimated_new_tokens":1315,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":21684,"ongoing_decode_tokens":0,"pending_prefill_tokens":21684,"num_requests":1,"active_p_offloads":0,"cached_blocks":1554,"cache_hit":0,"new_prefill":39203,"score_linear":21684.0,"score_lmetric":60887},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":137188,"ongoing_decode_tokens":0,"pending_prefill_tokens":95204,"num_requests":2,"active_p_offloads":0,"cached_blocks":1620,"cache_hit":0,"new_prefill":39203,"score_linear":137188.0,"score_lmetric":268814},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":21009,"ongoing_decode_tokens":0,"pending_prefill_tokens":21009,"num_requests":1,"active_p_offloads":0,"cached_blocks":2083,"cache_hit":37888,"new_prefill":1315,"score_linear":-16879.0,"score_lmetric":22324},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2010,"cache_hit":0,"new_prefill":39203,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":302125,"ongoing_decode_tokens":113401,"pending_prefill_tokens":5428,"num_requests":3,"active_p_offloads":0,"cached_blocks":2564,"cache_hit":0,"new_prefill":39203,"score_linear":302125.0,"score_lmetric":133893},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":203233,"ongoing_decode_tokens":135124,"pending_prefill_tokens":17421,"num_requests":3,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":39203,"score_linear":203233.0,"score_lmetric":169872},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":24442,"ongoing_decode_tokens":24442,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1385,"cache_hit":0,"new_prefill":39203,"score_linear":24442.0,"score_lmetric":39203},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":124090,"ongoing_decode_tokens":0,"pending_prefill_tokens":111290,"num_requests":1,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":0,"new_prefill":39203,"score_linear":124090.0,"score_lmetric":150493}],"chosen_score_linear":-16879.0,"chosen_score_lmetric":22324,"t_first_token":348896.4669263,"t_first_token_unix":1779779761.4808698,"t_done":348930.069272796,"t_done_unix":1779779795.083216},{"request_id":"1366128:23:1405900:781","session_id":"1366128","input_length":109574,"t_proxy_recv":348896.385162055,"t_decision_unix":1779779761.3991034,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":97792,"affinity_cache_ratio":0.8924744921240441,"affinity_num_requests":3,"avg_num_requests":2.25,"fallback_score":null,"tie_break_used":false,"cache_hit":97792,"estimated_new_tokens":11782,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":154908,"ongoing_decode_tokens":21684,"pending_prefill_tokens":12392,"num_requests":3,"active_p_offloads":0,"cached_blocks":1554,"cache_hit":0,"new_prefill":109574,"score_linear":154908.0,"score_lmetric":365898},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":137188,"ongoing_decode_tokens":93040,"pending_prefill_tokens":2676,"num_requests":2,"active_p_offloads":0,"cached_blocks":1620,"cache_hit":0,"new_prefill":109574,"score_linear":137188.0,"score_lmetric":224500},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":145650,"ongoing_decode_tokens":21009,"pending_prefill_tokens":5345,"num_requests":3,"active_p_offloads":0,"cached_blocks":2083,"cache_hit":97792,"new_prefill":11782,"score_linear":47858.0,"score_lmetric":51381},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":32944,"ongoing_decode_tokens":0,"pending_prefill_tokens":32944,"num_requests":1,"active_p_offloads":0,"cached_blocks":2010,"cache_hit":0,"new_prefill":109574,"score_linear":32944.0,"score_lmetric":142518},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":302125,"ongoing_decode_tokens":240668,"pending_prefill_tokens":4625,"num_requests":3,"active_p_offloads":0,"cached_blocks":2564,"cache_hit":0,"new_prefill":109574,"score_linear":302125.0,"score_lmetric":342597},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":203233,"ongoing_decode_tokens":203233,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":109574,"score_linear":203233.0,"score_lmetric":328722},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1408,"cache_hit":0,"new_prefill":109574,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":250216,"ongoing_decode_tokens":0,"pending_prefill_tokens":116584,"num_requests":3,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":0,"new_prefill":109574,"score_linear":250216.0,"score_lmetric":678474}],"chosen_score_linear":47858.0,"chosen_score_lmetric":51381,"t_first_token":348926.368277226,"t_first_token_unix":1779779791.3822205,"t_done":348931.417145262,"t_done_unix":1779779796.431089},{"request_id":"1340290:7:1408176:795","session_id":"1340290","input_length":27786,"t_proxy_recv":348904.414877503,"t_decision_unix":1779779769.4288185,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":27136,"affinity_cache_ratio":0.9766069243503923,"affinity_num_requests":4,"avg_num_requests":2.25,"fallback_score":null,"tie_break_used":false,"cache_hit":27136,"estimated_new_tokens":650,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":154908,"ongoing_decode_tokens":21684,"pending_prefill_tokens":12392,"num_requests":3,"active_p_offloads":0,"cached_blocks":1554,"cache_hit":0,"new_prefill":27786,"score_linear":154908.0,"score_lmetric":120534},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":137188,"ongoing_decode_tokens":137188,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1620,"cache_hit":0,"new_prefill":27786,"score_linear":137188.0,"score_lmetric":55572},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":255224,"ongoing_decode_tokens":60212,"pending_prefill_tokens":15812,"num_requests":4,"active_p_offloads":0,"cached_blocks":2083,"cache_hit":27136,"new_prefill":650,"score_linear":228088.0,"score_lmetric":65848},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2078,"cache_hit":0,"new_prefill":27786,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":302532,"ongoing_decode_tokens":190803,"pending_prefill_tokens":5233,"num_requests":3,"active_p_offloads":0,"cached_blocks":2564,"cache_hit":0,"new_prefill":27786,"score_linear":302532.0,"score_lmetric":99057},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":162800,"ongoing_decode_tokens":135124,"pending_prefill_tokens":5660,"num_requests":3,"active_p_offloads":0,"cached_blocks":1625,"cache_hit":0,"new_prefill":27786,"score_linear":162800.0,"score_lmetric":100338},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1425,"cache_hit":1024,"new_prefill":26762,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":250216,"ongoing_decode_tokens":0,"pending_prefill_tokens":116584,"num_requests":3,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":0,"new_prefill":27786,"score_linear":250216.0,"score_lmetric":433110}],"chosen_score_linear":228088.0,"chosen_score_lmetric":65848,"t_first_token":348929.837712566,"t_first_token_unix":1779779794.8516562,"t_done":348931.969927343,"t_done_unix":1779779796.9838707},{"request_id":"1277428:8:1412675:821","session_id":"1277428","input_length":40273,"t_proxy_recv":348919.855612633,"t_decision_unix":1779779784.8695538,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":38400,"affinity_cache_ratio":0.9534924142725896,"affinity_num_requests":3,"avg_num_requests":2.125,"fallback_score":null,"tie_break_used":false,"cache_hit":38400,"estimated_new_tokens":1873,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":154908,"ongoing_decode_tokens":21684,"pending_prefill_tokens":12392,"num_requests":3,"active_p_offloads":0,"cached_blocks":1554,"cache_hit":0,"new_prefill":40273,"score_linear":154908.0,"score_lmetric":157995},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1828,"cache_hit":4608,"new_prefill":35665,"score_linear":-4608.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":283010,"ongoing_decode_tokens":145650,"pending_prefill_tokens":12432,"num_requests":4,"active_p_offloads":0,"cached_blocks":2083,"cache_hit":0,"new_prefill":40273,"score_linear":283010.0,"score_lmetric":210820},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":107292,"ongoing_decode_tokens":0,"pending_prefill_tokens":14108,"num_requests":1,"active_p_offloads":0,"cached_blocks":2101,"cache_hit":17408,"new_prefill":22865,"score_linear":89884.0,"score_lmetric":36973},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":253194,"ongoing_decode_tokens":124993,"pending_prefill_tokens":1225,"num_requests":2,"active_p_offloads":0,"cached_blocks":2566,"cache_hit":0,"new_prefill":40273,"score_linear":253194.0,"score_lmetric":82996},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":207732,"ongoing_decode_tokens":207732,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1640,"cache_hit":38400,"new_prefill":1873,"score_linear":169332.0,"score_lmetric":5619},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1472,"cache_hit":0,"new_prefill":40273,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":323608,"ongoing_decode_tokens":0,"pending_prefill_tokens":127000,"num_requests":4,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":0,"new_prefill":40273,"score_linear":323608.0,"score_lmetric":669092}],"chosen_score_linear":169332.0,"chosen_score_lmetric":5619,"t_first_token":348926.308048701,"t_first_token_unix":1779779791.3219922,"t_done":348932.326987069,"t_done_unix":1779779797.3409305},{"request_id":"1366128:24:1407996:794","session_id":"1366128","input_length":109775,"t_proxy_recv":348931.434715228,"t_decision_unix":1779779796.4486566,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":109568,"affinity_cache_ratio":0.998114324755181,"affinity_num_requests":1,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"cache_hit":109568,"estimated_new_tokens":207,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":154908,"ongoing_decode_tokens":21684,"pending_prefill_tokens":12392,"num_requests":3,"active_p_offloads":0,"cached_blocks":1554,"cache_hit":0,"new_prefill":109775,"score_linear":154908.0,"score_lmetric":366501},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1871,"cache_hit":0,"new_prefill":109775,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":113224,"ongoing_decode_tokens":113224,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2149,"cache_hit":109568,"new_prefill":207,"score_linear":3656.0,"score_lmetric":207},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":147622,"ongoing_decode_tokens":0,"pending_prefill_tokens":21158,"num_requests":2,"active_p_offloads":0,"cached_blocks":2101,"cache_hit":0,"new_prefill":109775,"score_linear":147622.0,"score_lmetric":261866},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":124993,"ongoing_decode_tokens":124993,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2568,"cache_hit":0,"new_prefill":109775,"score_linear":124993.0,"score_lmetric":109775},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":248005,"ongoing_decode_tokens":248005,"pending_prefill_tokens":0,"num_requests":4,"active_p_offloads":0,"cached_blocks":1640,"cache_hit":0,"new_prefill":109775,"score_linear":248005.0,"score_lmetric":439100},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1503,"cache_hit":0,"new_prefill":109775,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":323608,"ongoing_decode_tokens":0,"pending_prefill_tokens":127000,"num_requests":4,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":0,"new_prefill":109775,"score_linear":323608.0,"score_lmetric":947100}],"chosen_score_linear":3656.0,"chosen_score_lmetric":207,"t_first_token":348931.695244394,"t_first_token_unix":1779779796.709189,"t_done":348932.76411775,"t_done_unix":1779779797.7780614},{"request_id":"1400379:4:1405483:779","session_id":"1400379","input_length":19966,"t_proxy_recv":348894.753827413,"t_decision_unix":1779779759.7677684,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":10752,"affinity_cache_ratio":0.5385154763097265,"affinity_num_requests":2,"avg_num_requests":2.25,"fallback_score":null,"tie_break_used":false,"cache_hit":10752,"estimated_new_tokens":9214,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":134942,"ongoing_decode_tokens":21684,"pending_prefill_tokens":3178,"num_requests":2,"active_p_offloads":0,"cached_blocks":1554,"cache_hit":10752,"new_prefill":9214,"score_linear":124190.0,"score_lmetric":24784},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":137188,"ongoing_decode_tokens":0,"pending_prefill_tokens":95204,"num_requests":2,"active_p_offloads":0,"cached_blocks":1620,"cache_hit":0,"new_prefill":19966,"score_linear":137188.0,"score_lmetric":230340},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":145650,"ongoing_decode_tokens":0,"pending_prefill_tokens":26354,"num_requests":3,"active_p_offloads":0,"cached_blocks":2083,"cache_hit":0,"new_prefill":19966,"score_linear":145650.0,"score_lmetric":138960},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":32944,"ongoing_decode_tokens":0,"pending_prefill_tokens":32944,"num_requests":1,"active_p_offloads":0,"cached_blocks":2010,"cache_hit":0,"new_prefill":19966,"score_linear":32944.0,"score_lmetric":52910},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":302125,"ongoing_decode_tokens":113401,"pending_prefill_tokens":5428,"num_requests":3,"active_p_offloads":0,"cached_blocks":2564,"cache_hit":0,"new_prefill":19966,"score_linear":302125.0,"score_lmetric":76182},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":203233,"ongoing_decode_tokens":203233,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":19966,"score_linear":203233.0,"score_lmetric":59898},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":35945,"ongoing_decode_tokens":24442,"pending_prefill_tokens":11503,"num_requests":2,"active_p_offloads":0,"cached_blocks":1385,"cache_hit":0,"new_prefill":19966,"score_linear":35945.0,"score_lmetric":62938},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":130239,"ongoing_decode_tokens":0,"pending_prefill_tokens":112831,"num_requests":2,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":3584,"new_prefill":16382,"score_linear":126655.0,"score_lmetric":258426}],"chosen_score_linear":124190.0,"chosen_score_lmetric":24784,"t_first_token":348936.396942048,"t_first_token_unix":1779779801.4108853,"t_done":348937.110321938,"t_done_unix":1779779802.1242654},{"request_id":"1366128:25:1411126:813","session_id":"1366128","input_length":117746,"t_proxy_recv":348932.786883191,"t_decision_unix":1779779797.8008246,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":109568,"affinity_cache_ratio":0.9305454113090891,"affinity_num_requests":1,"avg_num_requests":1.75,"fallback_score":null,"tie_break_used":false,"cache_hit":109568,"estimated_new_tokens":8178,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":154908,"ongoing_decode_tokens":21684,"pending_prefill_tokens":12392,"num_requests":3,"active_p_offloads":0,"cached_blocks":1554,"cache_hit":0,"new_prefill":117746,"score_linear":154908.0,"score_lmetric":390414},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1871,"cache_hit":0,"new_prefill":117746,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":115168,"ongoing_decode_tokens":115168,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2150,"cache_hit":109568,"new_prefill":8178,"score_linear":5600.0,"score_lmetric":8178},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":147622,"ongoing_decode_tokens":0,"pending_prefill_tokens":21158,"num_requests":2,"active_p_offloads":0,"cached_blocks":2101,"cache_hit":0,"new_prefill":117746,"score_linear":147622.0,"score_lmetric":277808},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":124993,"ongoing_decode_tokens":124993,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2568,"cache_hit":0,"new_prefill":117746,"score_linear":124993.0,"score_lmetric":117746},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":207732,"ongoing_decode_tokens":207732,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1644,"cache_hit":0,"new_prefill":117746,"score_linear":207732.0,"score_lmetric":353238},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1503,"cache_hit":0,"new_prefill":117746,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":323608,"ongoing_decode_tokens":0,"pending_prefill_tokens":127000,"num_requests":4,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":0,"new_prefill":117746,"score_linear":323608.0,"score_lmetric":978984}],"chosen_score_linear":5600.0,"chosen_score_lmetric":8178,"t_first_token":348938.331514362,"t_first_token_unix":1779779803.345458,"t_done":348939.148435832,"t_done_unix":1779779804.1623824},{"request_id":"1400379:5:1406380:784","session_id":"1400379","input_length":22009,"t_proxy_recv":348937.116563087,"t_decision_unix":1779779802.1305044,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":19968,"affinity_cache_ratio":0.9072652096869462,"affinity_num_requests":2,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"cache_hit":19968,"estimated_new_tokens":2041,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":134942,"ongoing_decode_tokens":134942,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1572,"cache_hit":19968,"new_prefill":2041,"score_linear":114974.0,"score_lmetric":4082},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1871,"cache_hit":0,"new_prefill":22009,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":232914,"ongoing_decode_tokens":115168,"pending_prefill_tokens":8178,"num_requests":2,"active_p_offloads":0,"cached_blocks":2150,"cache_hit":0,"new_prefill":22009,"score_linear":232914.0,"score_lmetric":60374},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":147622,"ongoing_decode_tokens":0,"pending_prefill_tokens":21158,"num_requests":2,"active_p_offloads":0,"cached_blocks":2101,"cache_hit":0,"new_prefill":22009,"score_linear":147622.0,"score_lmetric":86334},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":168756,"ongoing_decode_tokens":124993,"pending_prefill_tokens":2803,"num_requests":2,"active_p_offloads":0,"cached_blocks":2568,"cache_hit":0,"new_prefill":22009,"score_linear":168756.0,"score_lmetric":49624},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":207732,"ongoing_decode_tokens":207732,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1644,"cache_hit":0,"new_prefill":22009,"score_linear":207732.0,"score_lmetric":66027},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1503,"cache_hit":0,"new_prefill":22009,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":323608,"ongoing_decode_tokens":0,"pending_prefill_tokens":127000,"num_requests":4,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":3584,"new_prefill":18425,"score_linear":320024.0,"score_lmetric":581700}],"chosen_score_linear":114974.0,"chosen_score_lmetric":4082,"t_first_token":348937.520521721,"t_first_token_unix":1779779802.534465,"t_done":348940.366997441,"t_done_unix":1779779805.3809404},{"request_id":"1370129:3:1379538:637","session_id":"1370129","input_length":61457,"t_proxy_recv":348883.413986746,"t_decision_unix":1779779748.4279282,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":56832,"affinity_cache_ratio":0.9247441300421433,"affinity_num_requests":2,"avg_num_requests":2.0,"fallback_score":null,"tie_break_used":false,"cache_hit":56832,"estimated_new_tokens":4625,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":32332,"ongoing_decode_tokens":32332,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1530,"cache_hit":7680,"new_prefill":53777,"score_linear":24652.0,"score_lmetric":53777},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":93040,"ongoing_decode_tokens":0,"pending_prefill_tokens":92528,"num_requests":1,"active_p_offloads":0,"cached_blocks":1620,"cache_hit":0,"new_prefill":61457,"score_linear":93040.0,"score_lmetric":153985},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":124097,"ongoing_decode_tokens":26440,"pending_prefill_tokens":12153,"num_requests":2,"active_p_offloads":0,"cached_blocks":2036,"cache_hit":0,"new_prefill":61457,"score_linear":124097.0,"score_lmetric":147220},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":76392,"ongoing_decode_tokens":13134,"pending_prefill_tokens":12058,"num_requests":2,"active_p_offloads":0,"cached_blocks":1940,"cache_hit":0,"new_prefill":61457,"score_linear":76392.0,"score_lmetric":147030},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":240668,"ongoing_decode_tokens":113401,"pending_prefill_tokens":803,"num_requests":2,"active_p_offloads":0,"cached_blocks":2564,"cache_hit":56832,"new_prefill":4625,"score_linear":183836.0,"score_lmetric":10856},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":203233,"ongoing_decode_tokens":0,"pending_prefill_tokens":51169,"num_requests":3,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":61457,"score_linear":203233.0,"score_lmetric":337878},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":145452,"ongoing_decode_tokens":23641,"pending_prefill_tokens":59859,"num_requests":2,"active_p_offloads":0,"cached_blocks":1348,"cache_hit":7680,"new_prefill":53777,"score_linear":137772.0,"score_lmetric":227272},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":79403,"ongoing_decode_tokens":0,"pending_prefill_tokens":15403,"num_requests":3,"active_p_offloads":0,"cached_blocks":1342,"cache_hit":7680,"new_prefill":53777,"score_linear":71723.0,"score_lmetric":207540}],"chosen_score_linear":183836.0,"chosen_score_lmetric":10856,"t_first_token":348904.861205958,"t_first_token_unix":1779779769.8751493,"t_done":348943.440745299,"t_done_unix":1779779808.4546888},{"request_id":"1373577:3:1404917:774","session_id":"1373577","input_length":85438,"t_proxy_recv":348892.931336349,"t_decision_unix":1779779757.9452777,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":81408,"affinity_cache_ratio":0.9528312928673424,"affinity_num_requests":2,"avg_num_requests":1.625,"fallback_score":null,"tie_break_used":false,"cache_hit":81408,"estimated_new_tokens":4030,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":21684,"ongoing_decode_tokens":0,"pending_prefill_tokens":21684,"num_requests":1,"active_p_offloads":0,"cached_blocks":1554,"cache_hit":2560,"new_prefill":82878,"score_linear":19124.0,"score_lmetric":104562},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":137188,"ongoing_decode_tokens":0,"pending_prefill_tokens":95204,"num_requests":2,"active_p_offloads":0,"cached_blocks":1620,"cache_hit":0,"new_prefill":85438,"score_linear":137188.0,"score_lmetric":361284},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":60212,"ongoing_decode_tokens":0,"pending_prefill_tokens":22324,"num_requests":2,"active_p_offloads":0,"cached_blocks":2083,"cache_hit":81408,"new_prefill":4030,"score_linear":-21196.0,"score_lmetric":52708},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2010,"cache_hit":0,"new_prefill":85438,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":302125,"ongoing_decode_tokens":113401,"pending_prefill_tokens":5428,"num_requests":3,"active_p_offloads":0,"cached_blocks":2564,"cache_hit":0,"new_prefill":85438,"score_linear":302125.0,"score_lmetric":272598},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":203233,"ongoing_decode_tokens":135124,"pending_prefill_tokens":17421,"num_requests":3,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":85438,"score_linear":203233.0,"score_lmetric":308577},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":24442,"ongoing_decode_tokens":24442,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1385,"cache_hit":0,"new_prefill":85438,"score_linear":24442.0,"score_lmetric":85438},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":124090,"ongoing_decode_tokens":0,"pending_prefill_tokens":111290,"num_requests":1,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":0,"new_prefill":85438,"score_linear":124090.0,"score_lmetric":196728}],"chosen_score_linear":-21196.0,"chosen_score_lmetric":52708,"t_first_token":348919.475046827,"t_first_token_unix":1779779784.4889903,"t_done":348947.790506046,"t_done_unix":1779779812.8044503},{"request_id":"1362265:8:1419710:834","session_id":"1362265","input_length":11132,"t_proxy_recv":348944.645346674,"t_decision_unix":1779779809.6592877,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":10752,"affinity_cache_ratio":0.9658641753503413,"affinity_num_requests":1,"avg_num_requests":2.125,"fallback_score":null,"tie_break_used":false,"cache_hit":10752,"estimated_new_tokens":380,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":172340,"ongoing_decode_tokens":172340,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1576,"cache_hit":0,"new_prefill":11132,"score_linear":172340.0,"score_lmetric":33396},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1871,"cache_hit":0,"new_prefill":11132,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":243157,"ongoing_decode_tokens":115168,"pending_prefill_tokens":10229,"num_requests":2,"active_p_offloads":0,"cached_blocks":2166,"cache_hit":10752,"new_prefill":380,"score_linear":232405.0,"score_lmetric":21218},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":147622,"ongoing_decode_tokens":0,"pending_prefill_tokens":21158,"num_requests":2,"active_p_offloads":0,"cached_blocks":2101,"cache_hit":0,"new_prefill":11132,"score_linear":147622.0,"score_lmetric":64580},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":192483,"ongoing_decode_tokens":107299,"pending_prefill_tokens":23232,"num_requests":2,"active_p_offloads":0,"cached_blocks":2578,"cache_hit":0,"new_prefill":11132,"score_linear":192483.0,"score_lmetric":68728},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":207732,"ongoing_decode_tokens":207732,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1644,"cache_hit":0,"new_prefill":11132,"score_linear":207732.0,"score_lmetric":33396},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":32179,"ongoing_decode_tokens":0,"pending_prefill_tokens":25523,"num_requests":1,"active_p_offloads":0,"cached_blocks":1503,"cache_hit":10752,"new_prefill":380,"score_linear":21427.0,"score_lmetric":25903},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":323608,"ongoing_decode_tokens":130239,"pending_prefill_tokens":14169,"num_requests":4,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":0,"new_prefill":11132,"score_linear":323608.0,"score_lmetric":101204}],"chosen_score_linear":21427.0,"chosen_score_lmetric":25903,"t_first_token":348948.494764729,"t_first_token_unix":1779779813.5087078,"t_done":348948.829324429,"t_done_unix":1779779813.843268},{"request_id":"1340290:8:1409129:805","session_id":"1340290","input_length":29730,"t_proxy_recv":348931.977460019,"t_decision_unix":1779779796.9914017,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":27648,"affinity_cache_ratio":0.9299697275479314,"affinity_num_requests":1,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"cache_hit":27648,"estimated_new_tokens":2082,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":154908,"ongoing_decode_tokens":21684,"pending_prefill_tokens":12392,"num_requests":3,"active_p_offloads":0,"cached_blocks":1554,"cache_hit":0,"new_prefill":29730,"score_linear":154908.0,"score_lmetric":126366},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1871,"cache_hit":0,"new_prefill":29730,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":195213,"ongoing_decode_tokens":195213,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2150,"cache_hit":27648,"new_prefill":2082,"score_linear":167565.0,"score_lmetric":2082},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":147622,"ongoing_decode_tokens":0,"pending_prefill_tokens":21158,"num_requests":2,"active_p_offloads":0,"cached_blocks":2101,"cache_hit":0,"new_prefill":29730,"score_linear":147622.0,"score_lmetric":101776},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":124993,"ongoing_decode_tokens":124993,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2568,"cache_hit":0,"new_prefill":29730,"score_linear":124993.0,"score_lmetric":29730},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":248005,"ongoing_decode_tokens":248005,"pending_prefill_tokens":0,"num_requests":4,"active_p_offloads":0,"cached_blocks":1640,"cache_hit":0,"new_prefill":29730,"score_linear":248005.0,"score_lmetric":118920},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1503,"cache_hit":1024,"new_prefill":28706,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":323608,"ongoing_decode_tokens":0,"pending_prefill_tokens":127000,"num_requests":4,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":0,"new_prefill":29730,"score_linear":323608.0,"score_lmetric":626920}],"chosen_score_linear":167565.0,"chosen_score_lmetric":2082,"t_first_token":348932.477436538,"t_first_token_unix":1779779797.4913802,"t_done":348953.021592404,"t_done_unix":1779779818.0355382},{"request_id":"1362265:9:1422050:838","session_id":"1362265","input_length":11879,"t_proxy_recv":348953.106856301,"t_decision_unix":1779779818.1207979,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":10752,"affinity_cache_ratio":0.9051266941661756,"affinity_num_requests":1,"avg_num_requests":2.25,"fallback_score":null,"tie_break_used":false,"cache_hit":10752,"estimated_new_tokens":1127,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":172340,"ongoing_decode_tokens":172340,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1576,"cache_hit":0,"new_prefill":11879,"score_linear":172340.0,"score_lmetric":35637},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":49308,"ongoing_decode_tokens":49308,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1871,"cache_hit":0,"new_prefill":11879,"score_linear":49308.0,"score_lmetric":11879},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":248765,"ongoing_decode_tokens":218807,"pending_prefill_tokens":262,"num_requests":2,"active_p_offloads":0,"cached_blocks":2178,"cache_hit":10752,"new_prefill":1127,"score_linear":238013.0,"score_lmetric":2778},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":147622,"ongoing_decode_tokens":0,"pending_prefill_tokens":21158,"num_requests":2,"active_p_offloads":0,"cached_blocks":2101,"cache_hit":0,"new_prefill":11879,"score_linear":147622.0,"score_lmetric":66074},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":192483,"ongoing_decode_tokens":107299,"pending_prefill_tokens":23232,"num_requests":2,"active_p_offloads":0,"cached_blocks":2578,"cache_hit":0,"new_prefill":11879,"score_linear":192483.0,"score_lmetric":70222},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":207732,"ongoing_decode_tokens":207732,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1644,"cache_hit":0,"new_prefill":11879,"score_linear":207732.0,"score_lmetric":35637},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":32179,"ongoing_decode_tokens":32179,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1503,"cache_hit":10752,"new_prefill":1127,"score_linear":21427.0,"score_lmetric":1127},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":323608,"ongoing_decode_tokens":130239,"pending_prefill_tokens":14169,"num_requests":4,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":0,"new_prefill":11879,"score_linear":323608.0,"score_lmetric":104192}],"chosen_score_linear":21427.0,"chosen_score_lmetric":1127,"t_first_token":348953.245475692,"t_first_token_unix":1779779818.2594187,"t_done":348953.511531925,"t_done_unix":1779779818.5254755},{"request_id":"1268861:9:1421319:836","session_id":"1268861","input_length":49308,"t_proxy_recv":348950.375482776,"t_decision_unix":1779779815.3894238,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":44032,"affinity_cache_ratio":0.8929991076498742,"affinity_num_requests":0,"avg_num_requests":2.125,"fallback_score":null,"tie_break_used":false,"cache_hit":44032,"estimated_new_tokens":5276,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":172340,"ongoing_decode_tokens":172340,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1576,"cache_hit":0,"new_prefill":49308,"score_linear":172340.0,"score_lmetric":147924},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1871,"cache_hit":44032,"new_prefill":5276,"score_linear":-44032.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":248537,"ongoing_decode_tokens":157719,"pending_prefill_tokens":5314,"num_requests":2,"active_p_offloads":0,"cached_blocks":2174,"cache_hit":0,"new_prefill":49308,"score_linear":248537.0,"score_lmetric":109244},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":147622,"ongoing_decode_tokens":0,"pending_prefill_tokens":21158,"num_requests":2,"active_p_offloads":0,"cached_blocks":2101,"cache_hit":0,"new_prefill":49308,"score_linear":147622.0,"score_lmetric":140932},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":192483,"ongoing_decode_tokens":107299,"pending_prefill_tokens":23232,"num_requests":2,"active_p_offloads":0,"cached_blocks":2578,"cache_hit":0,"new_prefill":49308,"score_linear":192483.0,"score_lmetric":145080},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":207732,"ongoing_decode_tokens":207732,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1644,"cache_hit":0,"new_prefill":49308,"score_linear":207732.0,"score_lmetric":147924},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":32179,"ongoing_decode_tokens":32179,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1503,"cache_hit":0,"new_prefill":49308,"score_linear":32179.0,"score_lmetric":49308},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":323608,"ongoing_decode_tokens":130239,"pending_prefill_tokens":14169,"num_requests":4,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":12288,"new_prefill":37020,"score_linear":311320.0,"score_lmetric":204756}],"chosen_score_linear":-44032.0,"chosen_score_lmetric":0,"t_first_token":348951.949243516,"t_first_token_unix":1779779816.963187,"t_done":348953.6283149,"t_done_unix":1779779818.6422582},{"request_id":"1391404:2:1393919:720","session_id":"1391404","input_length":33321,"t_proxy_recv":348854.463826989,"t_decision_unix":1779779719.4777684,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":28160,"affinity_cache_ratio":0.8451126916959275,"affinity_num_requests":1,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":28160,"estimated_new_tokens":5161,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1502,"cache_hit":0,"new_prefill":33321,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":20683,"ongoing_decode_tokens":0,"pending_prefill_tokens":715,"num_requests":1,"active_p_offloads":0,"cached_blocks":1542,"cache_hit":0,"new_prefill":33321,"score_linear":20683.0,"score_lmetric":34036},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1982,"cache_hit":0,"new_prefill":33321,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1927,"cache_hit":0,"new_prefill":33321,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":182626,"pending_prefill_tokens":761,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":0,"new_prefill":33321,"score_linear":296027.0,"score_lmetric":102246},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":101803,"ongoing_decode_tokens":0,"pending_prefill_tokens":28587,"num_requests":1,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":28160,"new_prefill":5161,"score_linear":73643.0,"score_lmetric":33748},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":145452,"ongoing_decode_tokens":23641,"pending_prefill_tokens":59859,"num_requests":2,"active_p_offloads":0,"cached_blocks":1348,"cache_hit":0,"new_prefill":33321,"score_linear":145452.0,"score_lmetric":186360},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":147211,"ongoing_decode_tokens":147211,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1117,"cache_hit":0,"new_prefill":33321,"score_linear":147211.0,"score_lmetric":66642}],"chosen_score_linear":73643.0,"chosen_score_lmetric":33748,"t_first_token":348890.825113855,"t_first_token_unix":1779779755.8390572,"t_done":348954.018717523,"t_done_unix":1779779819.0326614},{"request_id":"1340290:9:1410595:811","session_id":"1340290","input_length":29958,"t_proxy_recv":348953.027650866,"t_decision_unix":1779779818.0415926,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":29696,"affinity_cache_ratio":0.9912544228586688,"affinity_num_requests":1,"avg_num_requests":2.125,"fallback_score":null,"tie_break_used":false,"cache_hit":29696,"estimated_new_tokens":262,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":172340,"ongoing_decode_tokens":172340,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1576,"cache_hit":0,"new_prefill":29958,"score_linear":172340.0,"score_lmetric":89874},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":49308,"ongoing_decode_tokens":49308,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1871,"cache_hit":0,"new_prefill":29958,"score_linear":49308.0,"score_lmetric":29958},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":218807,"ongoing_decode_tokens":218807,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2178,"cache_hit":29696,"new_prefill":262,"score_linear":189111.0,"score_lmetric":262},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":147622,"ongoing_decode_tokens":0,"pending_prefill_tokens":21158,"num_requests":2,"active_p_offloads":0,"cached_blocks":2101,"cache_hit":0,"new_prefill":29958,"score_linear":147622.0,"score_lmetric":102232},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":192483,"ongoing_decode_tokens":107299,"pending_prefill_tokens":23232,"num_requests":2,"active_p_offloads":0,"cached_blocks":2578,"cache_hit":0,"new_prefill":29958,"score_linear":192483.0,"score_lmetric":106380},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":207732,"ongoing_decode_tokens":207732,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1644,"cache_hit":0,"new_prefill":29958,"score_linear":207732.0,"score_lmetric":89874},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":32179,"ongoing_decode_tokens":32179,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1503,"cache_hit":1024,"new_prefill":28934,"score_linear":31155.0,"score_lmetric":28934},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":323608,"ongoing_decode_tokens":130239,"pending_prefill_tokens":14169,"num_requests":4,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":0,"new_prefill":29958,"score_linear":323608.0,"score_lmetric":176508}],"chosen_score_linear":189111.0,"chosen_score_lmetric":262,"t_first_token":348953.152066315,"t_first_token_unix":1779779818.1660094,"t_done":348954.404679744,"t_done_unix":1779779819.418623},{"request_id":"1404395:1:1404395:770","session_id":"1404395","input_length":21684,"t_proxy_recv":348891.029726952,"t_decision_unix":1779779756.0436683,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":null,"chosen_idx":0,"affinity_cache_hit":null,"affinity_cache_ratio":null,"affinity_num_requests":null,"avg_num_requests":1.375,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":21684,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1554,"cache_hit":0,"new_prefill":21684,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":137188,"ongoing_decode_tokens":0,"pending_prefill_tokens":95204,"num_requests":2,"active_p_offloads":0,"cached_blocks":1620,"cache_hit":0,"new_prefill":21684,"score_linear":137188.0,"score_lmetric":233776},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2083,"cache_hit":0,"new_prefill":21684,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":63643,"ongoing_decode_tokens":63643,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2009,"cache_hit":0,"new_prefill":21684,"score_linear":63643.0,"score_lmetric":21684},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":302125,"ongoing_decode_tokens":113401,"pending_prefill_tokens":5428,"num_requests":3,"active_p_offloads":0,"cached_blocks":2564,"cache_hit":0,"new_prefill":21684,"score_linear":302125.0,"score_lmetric":81336},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":203233,"ongoing_decode_tokens":135124,"pending_prefill_tokens":17421,"num_requests":3,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":512,"new_prefill":21172,"score_linear":202721.0,"score_lmetric":115779},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":86460,"ongoing_decode_tokens":86460,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1385,"cache_hit":0,"new_prefill":21684,"score_linear":86460.0,"score_lmetric":43368},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":0,"new_prefill":21684,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":348893.367797457,"t_first_token_unix":1779779758.3817408,"t_done":348955.282282596,"t_done_unix":1779779820.2962258},{"request_id":"1373577:4:1408642:799","session_id":"1373577","input_length":90818,"t_proxy_recv":348947.807940819,"t_decision_unix":1779779812.8218825,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":85504,"affinity_cache_ratio":0.9414873703450858,"affinity_num_requests":1,"avg_num_requests":2.125,"fallback_score":null,"tie_break_used":false,"cache_hit":85504,"estimated_new_tokens":5314,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":172340,"ongoing_decode_tokens":172340,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1576,"cache_hit":2560,"new_prefill":88258,"score_linear":169780.0,"score_lmetric":264774},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1871,"cache_hit":0,"new_prefill":90818,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":157719,"ongoing_decode_tokens":157719,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2174,"cache_hit":85504,"new_prefill":5314,"score_linear":72215.0,"score_lmetric":5314},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":147622,"ongoing_decode_tokens":0,"pending_prefill_tokens":21158,"num_requests":2,"active_p_offloads":0,"cached_blocks":2101,"cache_hit":0,"new_prefill":90818,"score_linear":147622.0,"score_lmetric":223952},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":192483,"ongoing_decode_tokens":107299,"pending_prefill_tokens":23232,"num_requests":2,"active_p_offloads":0,"cached_blocks":2578,"cache_hit":0,"new_prefill":90818,"score_linear":192483.0,"score_lmetric":228100},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":207732,"ongoing_decode_tokens":207732,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1644,"cache_hit":0,"new_prefill":90818,"score_linear":207732.0,"score_lmetric":272454},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":43311,"ongoing_decode_tokens":0,"pending_prefill_tokens":25903,"num_requests":2,"active_p_offloads":0,"cached_blocks":1503,"cache_hit":0,"new_prefill":90818,"score_linear":43311.0,"score_lmetric":233442},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":323608,"ongoing_decode_tokens":130239,"pending_prefill_tokens":14169,"num_requests":4,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":0,"new_prefill":90818,"score_linear":323608.0,"score_lmetric":419948}],"chosen_score_linear":72215.0,"chosen_score_lmetric":5314,"t_first_token":348950.720486194,"t_first_token_unix":1779779815.7344298,"t_done":348955.985174359,"t_done_unix":1779779820.9991179},{"request_id":"1264679:2:1405259:777","session_id":"1264679","input_length":113258,"t_proxy_recv":348894.06245515,"t_decision_unix":1779779759.0763965,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":110080,"affinity_cache_ratio":0.9719401719966801,"affinity_num_requests":1,"avg_num_requests":2.0,"fallback_score":null,"tie_break_used":false,"cache_hit":110080,"estimated_new_tokens":3178,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":21684,"ongoing_decode_tokens":21684,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1554,"cache_hit":110080,"new_prefill":3178,"score_linear":-88396.0,"score_lmetric":3178},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":137188,"ongoing_decode_tokens":0,"pending_prefill_tokens":95204,"num_requests":2,"active_p_offloads":0,"cached_blocks":1620,"cache_hit":0,"new_prefill":113258,"score_linear":137188.0,"score_lmetric":416924},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":145650,"ongoing_decode_tokens":0,"pending_prefill_tokens":26354,"num_requests":3,"active_p_offloads":0,"cached_blocks":2083,"cache_hit":0,"new_prefill":113258,"score_linear":145650.0,"score_lmetric":418836},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":32944,"ongoing_decode_tokens":0,"pending_prefill_tokens":32944,"num_requests":1,"active_p_offloads":0,"cached_blocks":2010,"cache_hit":0,"new_prefill":113258,"score_linear":32944.0,"score_lmetric":146202},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":302125,"ongoing_decode_tokens":113401,"pending_prefill_tokens":5428,"num_requests":3,"active_p_offloads":0,"cached_blocks":2564,"cache_hit":0,"new_prefill":113258,"score_linear":302125.0,"score_lmetric":356058},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":203233,"ongoing_decode_tokens":135124,"pending_prefill_tokens":17421,"num_requests":3,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":113258,"score_linear":203233.0,"score_lmetric":392037},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":35945,"ongoing_decode_tokens":24442,"pending_prefill_tokens":11503,"num_requests":2,"active_p_offloads":0,"cached_blocks":1385,"cache_hit":0,"new_prefill":113258,"score_linear":35945.0,"score_lmetric":249522},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":124090,"ongoing_decode_tokens":0,"pending_prefill_tokens":111290,"num_requests":1,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":0,"new_prefill":113258,"score_linear":124090.0,"score_lmetric":224548}],"chosen_score_linear":-88396.0,"chosen_score_lmetric":3178,"t_first_token":348935.499191582,"t_first_token_unix":1779779800.5131354,"t_done":348956.043904652,"t_done_unix":1779779821.0578477},{"request_id":"1366128:26:1413108:823","session_id":"1366128","input_length":127989,"t_proxy_recv":348939.176215993,"t_decision_unix":1779779804.1901577,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":117760,"affinity_cache_ratio":0.9200790692950176,"affinity_num_requests":1,"avg_num_requests":2.0,"fallback_score":null,"tie_break_used":false,"cache_hit":117760,"estimated_new_tokens":10229,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":194349,"ongoing_decode_tokens":156951,"pending_prefill_tokens":22,"num_requests":4,"active_p_offloads":0,"cached_blocks":1572,"cache_hit":0,"new_prefill":127989,"score_linear":194349.0,"score_lmetric":512044},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1871,"cache_hit":0,"new_prefill":127989,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":115168,"ongoing_decode_tokens":115168,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2166,"cache_hit":117760,"new_prefill":10229,"score_linear":-2592.0,"score_lmetric":10229},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":147622,"ongoing_decode_tokens":0,"pending_prefill_tokens":21158,"num_requests":2,"active_p_offloads":0,"cached_blocks":2101,"cache_hit":0,"new_prefill":127989,"score_linear":147622.0,"score_lmetric":298294},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":168756,"ongoing_decode_tokens":124993,"pending_prefill_tokens":2803,"num_requests":2,"active_p_offloads":0,"cached_blocks":2568,"cache_hit":0,"new_prefill":127989,"score_linear":168756.0,"score_lmetric":261584},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":207732,"ongoing_decode_tokens":207732,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1644,"cache_hit":0,"new_prefill":127989,"score_linear":207732.0,"score_lmetric":383967},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1503,"cache_hit":0,"new_prefill":127989,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":323608,"ongoing_decode_tokens":0,"pending_prefill_tokens":127000,"num_requests":4,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":0,"new_prefill":127989,"score_linear":323608.0,"score_lmetric":1019956}],"chosen_score_linear":-2592.0,"chosen_score_lmetric":10229,"t_first_token":348946.92893562,"t_first_token_unix":1779779811.942879,"t_done":348956.142431029,"t_done_unix":1779779821.1563747},{"request_id":"1270606:2:1411654:817","session_id":"1270606","input_length":107292,"t_proxy_recv":348916.539271151,"t_decision_unix":1779779781.5532124,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":93184,"affinity_cache_ratio":0.8685083696827349,"affinity_num_requests":0,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"cache_hit":93184,"estimated_new_tokens":14108,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":154908,"ongoing_decode_tokens":21684,"pending_prefill_tokens":12392,"num_requests":3,"active_p_offloads":0,"cached_blocks":1554,"cache_hit":0,"new_prefill":107292,"score_linear":154908.0,"score_lmetric":359052},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1828,"cache_hit":0,"new_prefill":107292,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":283010,"ongoing_decode_tokens":60212,"pending_prefill_tokens":16462,"num_requests":4,"active_p_offloads":0,"cached_blocks":2083,"cache_hit":0,"new_prefill":107292,"score_linear":283010.0,"score_lmetric":495016},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2101,"cache_hit":93184,"new_prefill":14108,"score_linear":-93184.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":253194,"ongoing_decode_tokens":124993,"pending_prefill_tokens":1225,"num_requests":2,"active_p_offloads":0,"cached_blocks":2566,"cache_hit":1536,"new_prefill":105756,"score_linear":251658.0,"score_lmetric":213962},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":135124,"ongoing_decode_tokens":135124,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1640,"cache_hit":0,"new_prefill":107292,"score_linear":135124.0,"score_lmetric":214584},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1472,"cache_hit":0,"new_prefill":107292,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":323608,"ongoing_decode_tokens":0,"pending_prefill_tokens":127000,"num_requests":4,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":0,"new_prefill":107292,"score_linear":323608.0,"score_lmetric":937168}],"chosen_score_linear":-93184.0,"chosen_score_lmetric":0,"t_first_token":348955.494974605,"t_first_token_unix":1779779820.508918,"t_done":348957.170628293,"t_done_unix":1779779822.184572},{"request_id":"1366128:27:1417788:831","session_id":"1366128","input_length":128433,"t_proxy_recv":348956.162346927,"t_decision_unix":1779779821.176288,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":128000,"affinity_cache_ratio":0.9966285923399749,"affinity_num_requests":0,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":128000,"estimated_new_tokens":433,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":37398,"ongoing_decode_tokens":37398,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":0,"new_prefill":128433,"score_linear":37398.0,"score_lmetric":128433},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1881,"cache_hit":0,"new_prefill":128433,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":30154,"ongoing_decode_tokens":30154,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2208,"cache_hit":128000,"new_prefill":433,"score_linear":-97846.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":147622,"ongoing_decode_tokens":147622,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":2101,"cache_hit":0,"new_prefill":128433,"score_linear":147622.0,"score_lmetric":256866},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":192483,"ongoing_decode_tokens":192483,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":2578,"cache_hit":0,"new_prefill":128433,"score_linear":192483.0,"score_lmetric":256866},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":174411,"ongoing_decode_tokens":174411,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1656,"cache_hit":0,"new_prefill":128433,"score_linear":174411.0,"score_lmetric":256866},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":32179,"ongoing_decode_tokens":32179,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1505,"cache_hit":0,"new_prefill":128433,"score_linear":32179.0,"score_lmetric":128433},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":323608,"ongoing_decode_tokens":130239,"pending_prefill_tokens":14169,"num_requests":4,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":0,"new_prefill":128433,"score_linear":323608.0,"score_lmetric":570408}],"chosen_score_linear":-97846.0,"chosen_score_lmetric":0,"t_first_token":348956.545772442,"t_first_token_unix":1779779821.559716,"t_done":348957.738752686,"t_done_unix":1779779822.7526965},{"request_id":"1366128:28:1421998:837","session_id":"1366128","input_length":128642,"t_proxy_recv":348957.760897261,"t_decision_unix":1779779822.7748387,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":128512,"affinity_cache_ratio":0.9989894435720837,"affinity_num_requests":0,"avg_num_requests":1.375,"fallback_score":null,"tie_break_used":false,"cache_hit":128512,"estimated_new_tokens":130,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":37398,"ongoing_decode_tokens":37398,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":0,"new_prefill":128642,"score_linear":37398.0,"score_lmetric":128642},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1881,"cache_hit":0,"new_prefill":128642,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":30154,"ongoing_decode_tokens":30154,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2209,"cache_hit":128512,"new_prefill":130,"score_linear":-98358.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":40330,"ongoing_decode_tokens":40330,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":0,"new_prefill":128642,"score_linear":40330.0,"score_lmetric":128642},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":192483,"ongoing_decode_tokens":192483,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":2578,"cache_hit":0,"new_prefill":128642,"score_linear":192483.0,"score_lmetric":257284},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":174411,"ongoing_decode_tokens":174411,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1656,"cache_hit":0,"new_prefill":128642,"score_linear":174411.0,"score_lmetric":257284},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":32179,"ongoing_decode_tokens":32179,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1505,"cache_hit":0,"new_prefill":128642,"score_linear":32179.0,"score_lmetric":128642},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":323608,"ongoing_decode_tokens":130239,"pending_prefill_tokens":14169,"num_requests":4,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":0,"new_prefill":128642,"score_linear":323608.0,"score_lmetric":571244}],"chosen_score_linear":-98358.0,"chosen_score_lmetric":0,"t_first_token":348958.034889672,"t_first_token_unix":1779779823.0488331,"t_done":348958.649738746,"t_done_unix":1779779823.6636825},{"request_id":"1340290:10:1411397:816","session_id":"1340290","input_length":30154,"t_proxy_recv":348954.41256394,"t_decision_unix":1779779819.4265053,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":29696,"affinity_cache_ratio":0.9848113019831531,"affinity_num_requests":1,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"cache_hit":29696,"estimated_new_tokens":458,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":172340,"ongoing_decode_tokens":172340,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1576,"cache_hit":0,"new_prefill":30154,"score_linear":172340.0,"score_lmetric":90462},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1881,"cache_hit":0,"new_prefill":30154,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":218807,"ongoing_decode_tokens":218807,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2178,"cache_hit":29696,"new_prefill":458,"score_linear":189111.0,"score_lmetric":458},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":147622,"ongoing_decode_tokens":0,"pending_prefill_tokens":21158,"num_requests":2,"active_p_offloads":0,"cached_blocks":2101,"cache_hit":0,"new_prefill":30154,"score_linear":147622.0,"score_lmetric":102624},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":192483,"ongoing_decode_tokens":192483,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":2578,"cache_hit":0,"new_prefill":30154,"score_linear":192483.0,"score_lmetric":60308},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":174411,"ongoing_decode_tokens":174411,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1656,"cache_hit":0,"new_prefill":30154,"score_linear":174411.0,"score_lmetric":60308},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":32179,"ongoing_decode_tokens":32179,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1505,"cache_hit":1024,"new_prefill":29130,"score_linear":31155.0,"score_lmetric":29130},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":323608,"ongoing_decode_tokens":130239,"pending_prefill_tokens":14169,"num_requests":4,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":0,"new_prefill":30154,"score_linear":323608.0,"score_lmetric":177292}],"chosen_score_linear":189111.0,"chosen_score_lmetric":458,"t_first_token":348954.546954804,"t_first_token_unix":1779779819.5608988,"t_done":348959.003441026,"t_done_unix":1779779824.0173843},{"request_id":"1364128:3:1419644:833","session_id":"1364128","input_length":32179,"t_proxy_recv":348944.418930511,"t_decision_unix":1779779809.4328716,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":2,"chosen_idx":6,"affinity_cache_hit":8704,"affinity_cache_ratio":0.2704869635476553,"affinity_num_requests":2,"avg_num_requests":2.0,"fallback_score":0,"tie_break_used":false,"cache_hit":6656,"estimated_new_tokens":25523,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":172340,"ongoing_decode_tokens":172340,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1576,"cache_hit":0,"new_prefill":32179,"score_linear":172340.0,"score_lmetric":96537},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1871,"cache_hit":0,"new_prefill":32179,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":243157,"ongoing_decode_tokens":115168,"pending_prefill_tokens":10229,"num_requests":2,"active_p_offloads":0,"cached_blocks":2166,"cache_hit":8704,"new_prefill":23475,"score_linear":234453.0,"score_lmetric":67408},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":147622,"ongoing_decode_tokens":0,"pending_prefill_tokens":21158,"num_requests":2,"active_p_offloads":0,"cached_blocks":2101,"cache_hit":0,"new_prefill":32179,"score_linear":147622.0,"score_lmetric":106674},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":192483,"ongoing_decode_tokens":107299,"pending_prefill_tokens":23232,"num_requests":2,"active_p_offloads":0,"cached_blocks":2578,"cache_hit":0,"new_prefill":32179,"score_linear":192483.0,"score_lmetric":110822},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":207732,"ongoing_decode_tokens":207732,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1644,"cache_hit":0,"new_prefill":32179,"score_linear":207732.0,"score_lmetric":96537},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1503,"cache_hit":6656,"new_prefill":25523,"score_linear":-6656.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":323608,"ongoing_decode_tokens":130239,"pending_prefill_tokens":14169,"num_requests":4,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":0,"new_prefill":32179,"score_linear":323608.0,"score_lmetric":185392}],"chosen_score_linear":-6656.0,"chosen_score_lmetric":0,"t_first_token":348948.491554244,"t_first_token_unix":1779779813.505498,"t_done":348959.367968126,"t_done_unix":1779779824.3819113},{"request_id":"1405155:2:1414214:828","session_id":"1405155","input_length":40330,"t_proxy_recv":348925.207753194,"t_decision_unix":1779779790.2216947,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":33280,"affinity_cache_ratio":0.825192164641706,"affinity_num_requests":1,"avg_num_requests":2.5,"fallback_score":null,"tie_break_used":false,"cache_hit":33280,"estimated_new_tokens":7050,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":154908,"ongoing_decode_tokens":21684,"pending_prefill_tokens":12392,"num_requests":3,"active_p_offloads":0,"cached_blocks":1554,"cache_hit":0,"new_prefill":40330,"score_linear":154908.0,"score_lmetric":158166},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":22392,"ongoing_decode_tokens":0,"pending_prefill_tokens":22392,"num_requests":1,"active_p_offloads":0,"cached_blocks":1828,"cache_hit":0,"new_prefill":40330,"score_linear":22392.0,"score_lmetric":62722},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":283010,"ongoing_decode_tokens":145650,"pending_prefill_tokens":12432,"num_requests":4,"active_p_offloads":0,"cached_blocks":2083,"cache_hit":0,"new_prefill":40330,"score_linear":283010.0,"score_lmetric":211048},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":107292,"ongoing_decode_tokens":0,"pending_prefill_tokens":14108,"num_requests":1,"active_p_offloads":0,"cached_blocks":2101,"cache_hit":33280,"new_prefill":7050,"score_linear":74012.0,"score_lmetric":21158},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":253194,"ongoing_decode_tokens":124993,"pending_prefill_tokens":1225,"num_requests":2,"active_p_offloads":0,"cached_blocks":2566,"cache_hit":0,"new_prefill":40330,"score_linear":253194.0,"score_lmetric":83110},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":248005,"ongoing_decode_tokens":207732,"pending_prefill_tokens":1873,"num_requests":4,"active_p_offloads":0,"cached_blocks":1640,"cache_hit":512,"new_prefill":39818,"score_linear":247493.0,"score_lmetric":166764},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":10633,"ongoing_decode_tokens":0,"pending_prefill_tokens":10633,"num_requests":1,"active_p_offloads":0,"cached_blocks":1482,"cache_hit":0,"new_prefill":40330,"score_linear":10633.0,"score_lmetric":50963},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":323608,"ongoing_decode_tokens":0,"pending_prefill_tokens":127000,"num_requests":4,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":0,"new_prefill":40330,"score_linear":323608.0,"score_lmetric":669320}],"chosen_score_linear":74012.0,"chosen_score_lmetric":21158,"t_first_token":348955.496879474,"t_first_token_unix":1779779820.5108228,"t_done":348960.061063449,"t_done_unix":1779779825.075007},{"request_id":"1301929:3:1417873:832","session_id":"1301929","input_length":37398,"t_proxy_recv":348938.238231988,"t_decision_unix":1779779803.2521734,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":37376,"affinity_cache_ratio":0.9994117332477672,"affinity_num_requests":3,"avg_num_requests":2.0,"fallback_score":null,"tie_break_used":false,"cache_hit":37376,"estimated_new_tokens":22,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":156951,"ongoing_decode_tokens":156951,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1572,"cache_hit":37376,"new_prefill":22,"score_linear":119575.0,"score_lmetric":66},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1871,"cache_hit":0,"new_prefill":37398,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":232914,"ongoing_decode_tokens":115168,"pending_prefill_tokens":8178,"num_requests":2,"active_p_offloads":0,"cached_blocks":2150,"cache_hit":0,"new_prefill":37398,"score_linear":232914.0,"score_lmetric":91152},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":147622,"ongoing_decode_tokens":0,"pending_prefill_tokens":21158,"num_requests":2,"active_p_offloads":0,"cached_blocks":2101,"cache_hit":0,"new_prefill":37398,"score_linear":147622.0,"score_lmetric":117112},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":168756,"ongoing_decode_tokens":124993,"pending_prefill_tokens":2803,"num_requests":2,"active_p_offloads":0,"cached_blocks":2568,"cache_hit":0,"new_prefill":37398,"score_linear":168756.0,"score_lmetric":80402},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":207732,"ongoing_decode_tokens":207732,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1644,"cache_hit":0,"new_prefill":37398,"score_linear":207732.0,"score_lmetric":112194},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1503,"cache_hit":0,"new_prefill":37398,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":323608,"ongoing_decode_tokens":0,"pending_prefill_tokens":127000,"num_requests":4,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":0,"new_prefill":37398,"score_linear":323608.0,"score_lmetric":657592}],"chosen_score_linear":119575.0,"chosen_score_lmetric":66,"t_first_token":348940.083505132,"t_first_token_unix":1779779805.0974486,"t_done":348962.105811701,"t_done_unix":1779779827.119755},{"request_id":"1340290:11:1413050:822","session_id":"1340290","input_length":35022,"t_proxy_recv":348959.013260043,"t_decision_unix":1779779824.0272012,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":30208,"affinity_cache_ratio":0.8625435440580207,"affinity_num_requests":-1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":30208,"estimated_new_tokens":4814,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":37398,"ongoing_decode_tokens":37398,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":0,"new_prefill":35022,"score_linear":37398.0,"score_lmetric":35022},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1881,"cache_hit":0,"new_prefill":35022,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":2210,"cache_hit":30208,"new_prefill":4814,"score_linear":-30208.0,"score_lmetric":-4814},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":40330,"ongoing_decode_tokens":40330,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2128,"cache_hit":0,"new_prefill":35022,"score_linear":40330.0,"score_lmetric":35022},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":192483,"ongoing_decode_tokens":192483,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":2578,"cache_hit":0,"new_prefill":35022,"score_linear":192483.0,"score_lmetric":70044},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":174411,"ongoing_decode_tokens":174411,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1656,"cache_hit":0,"new_prefill":35022,"score_linear":174411.0,"score_lmetric":70044},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":32179,"ongoing_decode_tokens":32179,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1505,"cache_hit":1024,"new_prefill":33998,"score_linear":31155.0,"score_lmetric":33998},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":323608,"ongoing_decode_tokens":130239,"pending_prefill_tokens":14169,"num_requests":4,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":0,"new_prefill":35022,"score_linear":323608.0,"score_lmetric":196764}],"chosen_score_linear":-30208.0,"chosen_score_lmetric":-4814,"t_first_token":348960.125632581,"t_first_token_unix":1779779825.139576,"t_done":348962.577875505,"t_done_unix":1779779827.591819},{"request_id":"1363943:11:1424840:842","session_id":"1363943","input_length":25225,"t_proxy_recv":348962.95154246,"t_decision_unix":1779779827.9654834,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":24576,"affinity_cache_ratio":0.9742715559960357,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":24576,"estimated_new_tokens":649,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":8704,"new_prefill":16521,"score_linear":-8704.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1881,"cache_hit":0,"new_prefill":25225,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":161501,"ongoing_decode_tokens":161501,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2219,"cache_hit":0,"new_prefill":25225,"score_linear":161501.0,"score_lmetric":25225},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2142,"cache_hit":0,"new_prefill":25225,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":192483,"ongoing_decode_tokens":192483,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":2578,"cache_hit":4096,"new_prefill":21129,"score_linear":188387.0,"score_lmetric":42258},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":174411,"ongoing_decode_tokens":174411,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1656,"cache_hit":0,"new_prefill":25225,"score_linear":174411.0,"score_lmetric":50450},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1555,"cache_hit":24576,"new_prefill":649,"score_linear":-24576.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":323608,"ongoing_decode_tokens":130239,"pending_prefill_tokens":14169,"num_requests":4,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":4096,"new_prefill":21129,"score_linear":319512.0,"score_lmetric":141192}],"chosen_score_linear":-24576.0,"chosen_score_lmetric":0,"t_first_token":348963.051258513,"t_first_token_unix":1779779828.0652018,"t_done":348964.035543844,"t_done_unix":1779779829.0494874},{"request_id":"1363093:3:1374394:612","session_id":"1363093","input_length":63536,"t_proxy_recv":348786.045255511,"t_decision_unix":1779779651.0591967,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":62976,"affinity_cache_ratio":0.9911860992193402,"affinity_num_requests":1,"avg_num_requests":1.5,"fallback_score":null,"tie_break_used":false,"cache_hit":62976,"estimated_new_tokens":560,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":182283,"ongoing_decode_tokens":98038,"pending_prefill_tokens":5909,"num_requests":3,"active_p_offloads":0,"cached_blocks":1381,"cache_hit":0,"new_prefill":63536,"score_linear":182283.0,"score_lmetric":208335},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":133759,"ongoing_decode_tokens":133759,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1008,"cache_hit":0,"new_prefill":63536,"score_linear":133759.0,"score_lmetric":127072},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126259,"ongoing_decode_tokens":0,"pending_prefill_tokens":91443,"num_requests":2,"active_p_offloads":0,"cached_blocks":1708,"cache_hit":0,"new_prefill":63536,"score_linear":126259.0,"score_lmetric":309958},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1750,"cache_hit":0,"new_prefill":63536,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":182626,"ongoing_decode_tokens":0,"pending_prefill_tokens":164194,"num_requests":1,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":62976,"new_prefill":560,"score_linear":119650.0,"score_lmetric":164754},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":19862,"ongoing_decode_tokens":0,"pending_prefill_tokens":19862,"num_requests":1,"active_p_offloads":0,"cached_blocks":1214,"cache_hit":0,"new_prefill":63536,"score_linear":19862.0,"score_lmetric":83398},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":105902,"ongoing_decode_tokens":0,"pending_prefill_tokens":45486,"num_requests":1,"active_p_offloads":0,"cached_blocks":1249,"cache_hit":0,"new_prefill":63536,"score_linear":105902.0,"score_lmetric":109022},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":106951,"ongoing_decode_tokens":106951,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1047,"cache_hit":0,"new_prefill":63536,"score_linear":106951.0,"score_lmetric":127072}],"chosen_score_linear":119650.0,"chosen_score_lmetric":164754,"t_first_token":348857.856735986,"t_first_token_unix":1779779722.8706796,"t_done":348964.069291293,"t_done_unix":1779779829.0832348},{"request_id":"1370129:4:1384856:678","session_id":"1370129","input_length":85184,"t_proxy_recv":348943.464192179,"t_decision_unix":1779779808.4781332,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":61952,"affinity_cache_ratio":0.7272727272727273,"affinity_num_requests":1,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"cache_hit":61952,"estimated_new_tokens":23232,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":172340,"ongoing_decode_tokens":172340,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1576,"cache_hit":7680,"new_prefill":77504,"score_linear":164660.0,"score_lmetric":232512},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1871,"cache_hit":0,"new_prefill":85184,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":243157,"ongoing_decode_tokens":115168,"pending_prefill_tokens":10229,"num_requests":2,"active_p_offloads":0,"cached_blocks":2166,"cache_hit":0,"new_prefill":85184,"score_linear":243157.0,"score_lmetric":190826},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":147622,"ongoing_decode_tokens":0,"pending_prefill_tokens":21158,"num_requests":2,"active_p_offloads":0,"cached_blocks":2101,"cache_hit":0,"new_prefill":85184,"score_linear":147622.0,"score_lmetric":212684},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":107299,"ongoing_decode_tokens":107299,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2578,"cache_hit":61952,"new_prefill":23232,"score_linear":45347.0,"score_lmetric":23232},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":207732,"ongoing_decode_tokens":207732,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1644,"cache_hit":0,"new_prefill":85184,"score_linear":207732.0,"score_lmetric":255552},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1503,"cache_hit":7680,"new_prefill":77504,"score_linear":-7680.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":323608,"ongoing_decode_tokens":130239,"pending_prefill_tokens":14169,"num_requests":4,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":7680,"new_prefill":77504,"score_linear":315928.0,"score_lmetric":366692}],"chosen_score_linear":45347.0,"chosen_score_lmetric":23232,"t_first_token":348954.236280445,"t_first_token_unix":1779779819.2502239,"t_done":348965.746869598,"t_done_unix":1779779830.7608128},{"request_id":"1355951:5:1425005:843","session_id":"1355951","input_length":39789,"t_proxy_recv":348963.601468746,"t_decision_unix":1779779828.6154099,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":37888,"affinity_cache_ratio":0.9522229761994521,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":37888,"estimated_new_tokens":1901,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":0,"new_prefill":39789,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1881,"cache_hit":0,"new_prefill":39789,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":161501,"ongoing_decode_tokens":161501,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2219,"cache_hit":37888,"new_prefill":1901,"score_linear":123613.0,"score_lmetric":1901},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2142,"cache_hit":0,"new_prefill":39789,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":192483,"ongoing_decode_tokens":192483,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":2578,"cache_hit":0,"new_prefill":39789,"score_linear":192483.0,"score_lmetric":79578},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":174411,"ongoing_decode_tokens":174411,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1656,"cache_hit":0,"new_prefill":39789,"score_linear":174411.0,"score_lmetric":79578},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":25225,"ongoing_decode_tokens":25225,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1555,"cache_hit":0,"new_prefill":39789,"score_linear":25225.0,"score_lmetric":39789},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":323608,"ongoing_decode_tokens":130239,"pending_prefill_tokens":14169,"num_requests":4,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":0,"new_prefill":39789,"score_linear":323608.0,"score_lmetric":215832}],"chosen_score_linear":123613.0,"chosen_score_lmetric":1901,"t_first_token":348966.882561645,"t_first_token_unix":1779779831.896505,"t_done":348967.431964777,"t_done_unix":1779779832.4459083},{"request_id":"1340290:12:1421294:835","session_id":"1340290","input_length":35303,"t_proxy_recv":348962.585093471,"t_decision_unix":1779779827.599035,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":34816,"affinity_cache_ratio":0.9862051383735093,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":34816,"estimated_new_tokens":487,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":0,"new_prefill":35303,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1881,"cache_hit":0,"new_prefill":35303,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126198,"ongoing_decode_tokens":126198,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2219,"cache_hit":34816,"new_prefill":487,"score_linear":91382.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2142,"cache_hit":0,"new_prefill":35303,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":192483,"ongoing_decode_tokens":192483,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":2578,"cache_hit":0,"new_prefill":35303,"score_linear":192483.0,"score_lmetric":70606},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":174411,"ongoing_decode_tokens":174411,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1656,"cache_hit":0,"new_prefill":35303,"score_linear":174411.0,"score_lmetric":70606},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1555,"cache_hit":1024,"new_prefill":34279,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":323608,"ongoing_decode_tokens":130239,"pending_prefill_tokens":14169,"num_requests":4,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":0,"new_prefill":35303,"score_linear":323608.0,"score_lmetric":197888}],"chosen_score_linear":91382.0,"chosen_score_lmetric":0,"t_first_token":348962.666799087,"t_first_token_unix":1779779827.6807425,"t_done":348969.167772423,"t_done_unix":1779779834.1817157},{"request_id":"1270606:3:1427164:848","session_id":"1270606","input_length":108318,"t_proxy_recv":348971.279863543,"t_decision_unix":1779779836.293805,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":107008,"affinity_cache_ratio":0.987905980538784,"affinity_num_requests":0,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":107008,"estimated_new_tokens":1310,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":0,"new_prefill":108318,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1881,"cache_hit":0,"new_prefill":108318,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126198,"ongoing_decode_tokens":126198,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":108318,"score_linear":126198.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2142,"cache_hit":107008,"new_prefill":1310,"score_linear":-107008.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199637,"ongoing_decode_tokens":199637,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":2627,"cache_hit":1536,"new_prefill":106782,"score_linear":198101.0,"score_lmetric":213564},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":216753,"ongoing_decode_tokens":216753,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1656,"cache_hit":0,"new_prefill":108318,"score_linear":216753.0,"score_lmetric":324954},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":26923,"ongoing_decode_tokens":26923,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1556,"cache_hit":0,"new_prefill":108318,"score_linear":26923.0,"score_lmetric":108318},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":323608,"ongoing_decode_tokens":130239,"pending_prefill_tokens":14169,"num_requests":4,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":0,"new_prefill":108318,"score_linear":323608.0,"score_lmetric":489948}],"chosen_score_linear":-107008.0,"chosen_score_lmetric":0,"t_first_token":348972.015105094,"t_first_token_unix":1779779837.0290482,"t_done":348972.309379513,"t_done_unix":1779779837.3233225},{"request_id":"1399948:2:1405276:778","session_id":"1399948","input_length":6149,"t_proxy_recv":348894.116705263,"t_decision_unix":1779779759.1306465,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":4608,"affinity_cache_ratio":0.7493901447389819,"affinity_num_requests":1,"avg_num_requests":2.125,"fallback_score":null,"tie_break_used":false,"cache_hit":4608,"estimated_new_tokens":1541,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":134942,"ongoing_decode_tokens":21684,"pending_prefill_tokens":3178,"num_requests":2,"active_p_offloads":0,"cached_blocks":1554,"cache_hit":0,"new_prefill":6149,"score_linear":134942.0,"score_lmetric":18654},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":137188,"ongoing_decode_tokens":0,"pending_prefill_tokens":95204,"num_requests":2,"active_p_offloads":0,"cached_blocks":1620,"cache_hit":0,"new_prefill":6149,"score_linear":137188.0,"score_lmetric":202706},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":145650,"ongoing_decode_tokens":0,"pending_prefill_tokens":26354,"num_requests":3,"active_p_offloads":0,"cached_blocks":2083,"cache_hit":0,"new_prefill":6149,"score_linear":145650.0,"score_lmetric":97509},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":32944,"ongoing_decode_tokens":0,"pending_prefill_tokens":32944,"num_requests":1,"active_p_offloads":0,"cached_blocks":2010,"cache_hit":0,"new_prefill":6149,"score_linear":32944.0,"score_lmetric":39093},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":302125,"ongoing_decode_tokens":113401,"pending_prefill_tokens":5428,"num_requests":3,"active_p_offloads":0,"cached_blocks":2564,"cache_hit":0,"new_prefill":6149,"score_linear":302125.0,"score_lmetric":34731},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":203233,"ongoing_decode_tokens":135124,"pending_prefill_tokens":17421,"num_requests":3,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":6149,"score_linear":203233.0,"score_lmetric":70710},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":35945,"ongoing_decode_tokens":24442,"pending_prefill_tokens":11503,"num_requests":2,"active_p_offloads":0,"cached_blocks":1385,"cache_hit":0,"new_prefill":6149,"score_linear":35945.0,"score_lmetric":35304},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":124090,"ongoing_decode_tokens":0,"pending_prefill_tokens":111290,"num_requests":1,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":4608,"new_prefill":1541,"score_linear":119482.0,"score_lmetric":112831}],"chosen_score_linear":119482.0,"chosen_score_lmetric":112831,"t_first_token":348939.665447695,"t_first_token_unix":1779779804.6793907,"t_done":348972.48289072,"t_done_unix":1779779837.4968338},{"request_id":"1366128:29:1424316:841","session_id":"1366128","input_length":126198,"t_proxy_recv":348961.130471116,"t_decision_unix":1779779826.1444125,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":125952,"affinity_cache_ratio":0.9980506822612085,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":125952,"estimated_new_tokens":246,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":37398,"ongoing_decode_tokens":37398,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":0,"new_prefill":126198,"score_linear":37398.0,"score_lmetric":126198},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1881,"cache_hit":0,"new_prefill":126198,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":35022,"ongoing_decode_tokens":35022,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2210,"cache_hit":125952,"new_prefill":246,"score_linear":-90930.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2142,"cache_hit":0,"new_prefill":126198,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":192483,"ongoing_decode_tokens":192483,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":2578,"cache_hit":0,"new_prefill":126198,"score_linear":192483.0,"score_lmetric":252396},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":174411,"ongoing_decode_tokens":174411,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1656,"cache_hit":0,"new_prefill":126198,"score_linear":174411.0,"score_lmetric":252396},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1555,"cache_hit":0,"new_prefill":126198,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":323608,"ongoing_decode_tokens":130239,"pending_prefill_tokens":14169,"num_requests":4,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":0,"new_prefill":126198,"score_linear":323608.0,"score_lmetric":561468}],"chosen_score_linear":-90930.0,"chosen_score_lmetric":0,"t_first_token":348961.278407315,"t_first_token_unix":1779779826.2923508,"t_done":348973.513262052,"t_done_unix":1779779838.527206},{"request_id":"1399948:3:1405988:782","session_id":"1399948","input_length":7995,"t_proxy_recv":348972.487131544,"t_decision_unix":1779779837.5010731,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":7,"chosen_idx":2,"affinity_cache_hit":6144,"affinity_cache_ratio":0.7684803001876173,"affinity_num_requests":3,"avg_num_requests":1.125,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":7995,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":0,"new_prefill":7995,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1881,"cache_hit":0,"new_prefill":7995,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126198,"ongoing_decode_tokens":126198,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":7995,"score_linear":126198.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2144,"cache_hit":0,"new_prefill":7995,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199637,"ongoing_decode_tokens":199637,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":2627,"cache_hit":0,"new_prefill":7995,"score_linear":199637.0,"score_lmetric":15990},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":216753,"ongoing_decode_tokens":216753,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1656,"cache_hit":0,"new_prefill":7995,"score_linear":216753.0,"score_lmetric":23985},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":26923,"ongoing_decode_tokens":26923,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1556,"cache_hit":0,"new_prefill":7995,"score_linear":26923.0,"score_lmetric":7995},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":317459,"ongoing_decode_tokens":124090,"pending_prefill_tokens":14169,"num_requests":3,"active_p_offloads":0,"cached_blocks":1376,"cache_hit":6144,"new_prefill":1851,"score_linear":311315.0,"score_lmetric":48060}],"chosen_score_linear":126198.0,"chosen_score_lmetric":0,"t_first_token":348973.084244035,"t_first_token_unix":1779779838.0981877,"t_done":348973.625037865,"t_done_unix":1779779838.6389813},{"request_id":"1399948:4:1407140:786","session_id":"1399948","input_length":8540,"t_proxy_recv":348973.628671655,"t_decision_unix":1779779838.6426132,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":7680,"affinity_cache_ratio":0.8992974238875878,"affinity_num_requests":-1,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":7680,"estimated_new_tokens":860,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":0,"new_prefill":8540,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1881,"cache_hit":0,"new_prefill":8540,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":2238,"cache_hit":7680,"new_prefill":860,"score_linear":-7680.0,"score_lmetric":-860},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2144,"cache_hit":0,"new_prefill":8540,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199637,"ongoing_decode_tokens":199637,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":2627,"cache_hit":0,"new_prefill":8540,"score_linear":199637.0,"score_lmetric":17080},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":216753,"ongoing_decode_tokens":216753,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1656,"cache_hit":0,"new_prefill":8540,"score_linear":216753.0,"score_lmetric":25620},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":39519,"ongoing_decode_tokens":39519,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1556,"cache_hit":0,"new_prefill":8540,"score_linear":39519.0,"score_lmetric":17080},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":317459,"ongoing_decode_tokens":124090,"pending_prefill_tokens":14169,"num_requests":3,"active_p_offloads":0,"cached_blocks":1376,"cache_hit":6144,"new_prefill":2396,"score_linear":311315.0,"score_lmetric":49695}],"chosen_score_linear":-7680.0,"chosen_score_lmetric":-860,"t_first_token":348973.714150623,"t_first_token_unix":1779779838.7280946,"t_done":348973.826134052,"t_done_unix":1779779838.8400807},{"request_id":"1363943:12:1426646:845","session_id":"1363943","input_length":26923,"t_proxy_recv":348969.424353219,"t_decision_unix":1779779834.4382942,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":25088,"affinity_cache_ratio":0.9318426624076068,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":25088,"estimated_new_tokens":1835,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":8704,"new_prefill":18219,"score_linear":-8704.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1881,"cache_hit":0,"new_prefill":26923,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126198,"ongoing_decode_tokens":126198,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":26923,"score_linear":126198.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2142,"cache_hit":0,"new_prefill":26923,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199637,"ongoing_decode_tokens":199637,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":2627,"cache_hit":4096,"new_prefill":22827,"score_linear":195541.0,"score_lmetric":45654},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":174411,"ongoing_decode_tokens":174411,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1656,"cache_hit":0,"new_prefill":26923,"score_linear":174411.0,"score_lmetric":53846},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1556,"cache_hit":25088,"new_prefill":1835,"score_linear":-25088.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":323608,"ongoing_decode_tokens":130239,"pending_prefill_tokens":14169,"num_requests":4,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":4096,"new_prefill":22827,"score_linear":319512.0,"score_lmetric":147984}],"chosen_score_linear":-25088.0,"chosen_score_lmetric":0,"t_first_token":348969.775226743,"t_first_token_unix":1779779834.7891705,"t_done":348973.966275768,"t_done_unix":1779779838.980219},{"request_id":"1399948:5:1407922:791","session_id":"1399948","input_length":8612,"t_proxy_recv":348973.82950899,"t_decision_unix":1779779838.8434505,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":8192,"affinity_cache_ratio":0.9512308406874129,"affinity_num_requests":-1,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":8192,"estimated_new_tokens":420,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":0,"new_prefill":8612,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1881,"cache_hit":0,"new_prefill":8612,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":2239,"cache_hit":8192,"new_prefill":420,"score_linear":-8192.0,"score_lmetric":-420},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2144,"cache_hit":0,"new_prefill":8612,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199637,"ongoing_decode_tokens":199637,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":2627,"cache_hit":0,"new_prefill":8612,"score_linear":199637.0,"score_lmetric":17224},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":216753,"ongoing_decode_tokens":216753,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1656,"cache_hit":0,"new_prefill":8612,"score_linear":216753.0,"score_lmetric":25836},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":39519,"ongoing_decode_tokens":39519,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1556,"cache_hit":0,"new_prefill":8612,"score_linear":39519.0,"score_lmetric":17224},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":317459,"ongoing_decode_tokens":124090,"pending_prefill_tokens":14169,"num_requests":3,"active_p_offloads":0,"cached_blocks":1376,"cache_hit":6144,"new_prefill":2468,"score_linear":311315.0,"score_lmetric":49911}],"chosen_score_linear":-8192.0,"chosen_score_lmetric":-420,"t_first_token":348973.863928009,"t_first_token_unix":1779779838.8778713,"t_done":348973.982561708,"t_done_unix":1779779838.9965048},{"request_id":"1363093:4:1390050:704","session_id":"1363093","input_length":68187,"t_proxy_recv":348964.084021903,"t_decision_unix":1779779829.097963,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":65024,"affinity_cache_ratio":0.9536128587560678,"affinity_num_requests":1,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":65024,"estimated_new_tokens":3163,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":0,"new_prefill":68187,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1881,"cache_hit":0,"new_prefill":68187,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":201290,"ongoing_decode_tokens":161501,"pending_prefill_tokens":1901,"num_requests":2,"active_p_offloads":0,"cached_blocks":2219,"cache_hit":0,"new_prefill":68187,"score_linear":201290.0,"score_lmetric":140176},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2142,"cache_hit":0,"new_prefill":68187,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":128947,"ongoing_decode_tokens":128947,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2581,"cache_hit":65024,"new_prefill":3163,"score_linear":63923.0,"score_lmetric":3163},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":174411,"ongoing_decode_tokens":174411,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1656,"cache_hit":0,"new_prefill":68187,"score_linear":174411.0,"score_lmetric":136374},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1556,"cache_hit":0,"new_prefill":68187,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":323608,"ongoing_decode_tokens":130239,"pending_prefill_tokens":14169,"num_requests":4,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":0,"new_prefill":68187,"score_linear":323608.0,"score_lmetric":329424}],"chosen_score_linear":63923.0,"chosen_score_lmetric":3163,"t_first_token":348965.450068882,"t_first_token_unix":1779779830.4640124,"t_done":348974.13169959,"t_done_unix":1779779839.1456432},{"request_id":"1314357:5:1393659:719","session_id":"1314357","input_length":101803,"t_proxy_recv":348853.526162603,"t_decision_unix":1779779718.5401037,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":73216,"affinity_cache_ratio":0.7191929510918146,"affinity_num_requests":0,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":73216,"estimated_new_tokens":28587,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1502,"cache_hit":0,"new_prefill":101803,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":92082,"ongoing_decode_tokens":92082,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1400,"cache_hit":0,"new_prefill":101803,"score_linear":92082.0,"score_lmetric":203606},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1982,"cache_hit":0,"new_prefill":101803,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1927,"cache_hit":0,"new_prefill":101803,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":296027,"ongoing_decode_tokens":182626,"pending_prefill_tokens":761,"num_requests":3,"active_p_offloads":0,"cached_blocks":2242,"cache_hit":56832,"new_prefill":44971,"score_linear":239195.0,"score_lmetric":137196},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":73216,"new_prefill":28587,"score_linear":-73216.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":145452,"ongoing_decode_tokens":23641,"pending_prefill_tokens":59859,"num_requests":2,"active_p_offloads":0,"cached_blocks":1348,"cache_hit":0,"new_prefill":101803,"score_linear":145452.0,"score_lmetric":323324},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":147211,"ongoing_decode_tokens":147211,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1117,"cache_hit":0,"new_prefill":101803,"score_linear":147211.0,"score_lmetric":203606}],"chosen_score_linear":-73216.0,"chosen_score_lmetric":0,"t_first_token":348887.987737537,"t_first_token_unix":1779779753.001681,"t_done":348974.341778912,"t_done_unix":1779779839.3557222},{"request_id":"1399948:6:1410696:812","session_id":"1399948","input_length":11381,"t_proxy_recv":348973.987434176,"t_decision_unix":1779779839.0013757,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":8192,"affinity_cache_ratio":0.7197961514805378,"affinity_num_requests":-1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":8192,"estimated_new_tokens":3189,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":0,"new_prefill":11381,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1881,"cache_hit":0,"new_prefill":11381,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":2239,"cache_hit":8192,"new_prefill":3189,"score_linear":-8192.0,"score_lmetric":-3189},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2144,"cache_hit":0,"new_prefill":11381,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199637,"ongoing_decode_tokens":199637,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":2627,"cache_hit":0,"new_prefill":11381,"score_linear":199637.0,"score_lmetric":22762},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":216753,"ongoing_decode_tokens":216753,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1656,"cache_hit":0,"new_prefill":11381,"score_linear":216753.0,"score_lmetric":34143},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":12596,"ongoing_decode_tokens":12596,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1560,"cache_hit":0,"new_prefill":11381,"score_linear":12596.0,"score_lmetric":11381},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":317459,"ongoing_decode_tokens":124090,"pending_prefill_tokens":14169,"num_requests":3,"active_p_offloads":0,"cached_blocks":1376,"cache_hit":6144,"new_prefill":5237,"score_linear":311315.0,"score_lmetric":58218}],"chosen_score_linear":-8192.0,"chosen_score_lmetric":-3189,"t_first_token":348974.301444254,"t_first_token_unix":1779779839.3153877,"t_done":348974.388283289,"t_done_unix":1779779839.4022264},{"request_id":"1399948:7:1413205:824","session_id":"1399948","input_length":11537,"t_proxy_recv":348974.391715968,"t_decision_unix":1779779839.4056575,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":11264,"affinity_cache_ratio":0.9763370026870071,"affinity_num_requests":-1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":11264,"estimated_new_tokens":273,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":0,"new_prefill":11537,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1881,"cache_hit":0,"new_prefill":11537,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":2245,"cache_hit":11264,"new_prefill":273,"score_linear":-11264.0,"score_lmetric":-273},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2144,"cache_hit":0,"new_prefill":11537,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":204638,"ongoing_decode_tokens":131450,"pending_prefill_tokens":4580,"num_requests":2,"active_p_offloads":0,"cached_blocks":2634,"cache_hit":0,"new_prefill":11537,"score_linear":204638.0,"score_lmetric":32234},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":220659,"ongoing_decode_tokens":114950,"pending_prefill_tokens":237,"num_requests":3,"active_p_offloads":0,"cached_blocks":1712,"cache_hit":0,"new_prefill":11537,"score_linear":220659.0,"score_lmetric":35322},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":12596,"ongoing_decode_tokens":12596,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1560,"cache_hit":0,"new_prefill":11537,"score_linear":12596.0,"score_lmetric":11537},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":317459,"ongoing_decode_tokens":124090,"pending_prefill_tokens":14169,"num_requests":3,"active_p_offloads":0,"cached_blocks":1376,"cache_hit":6144,"new_prefill":5393,"score_linear":311315.0,"score_lmetric":58686}],"chosen_score_linear":-11264.0,"chosen_score_lmetric":-273,"t_first_token":348974.450299158,"t_first_token_unix":1779779839.4642425,"t_done":348974.681365161,"t_done_unix":1779779839.6953084},{"request_id":"1399948:8:1425345:844","session_id":"1399948","input_length":11736,"t_proxy_recv":348974.685300642,"t_decision_unix":1779779839.6992424,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":11264,"affinity_cache_ratio":0.9597818677573279,"affinity_num_requests":-1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":11264,"estimated_new_tokens":472,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":0,"new_prefill":11736,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1881,"cache_hit":0,"new_prefill":11736,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":2245,"cache_hit":11264,"new_prefill":472,"score_linear":-11264.0,"score_lmetric":-472},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2144,"cache_hit":0,"new_prefill":11736,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":204638,"ongoing_decode_tokens":131450,"pending_prefill_tokens":4580,"num_requests":2,"active_p_offloads":0,"cached_blocks":2634,"cache_hit":0,"new_prefill":11736,"score_linear":204638.0,"score_lmetric":32632},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":220659,"ongoing_decode_tokens":220659,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1712,"cache_hit":0,"new_prefill":11736,"score_linear":220659.0,"score_lmetric":35208},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":12596,"ongoing_decode_tokens":12596,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1560,"cache_hit":0,"new_prefill":11736,"score_linear":12596.0,"score_lmetric":11736},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":317459,"ongoing_decode_tokens":124090,"pending_prefill_tokens":14169,"num_requests":3,"active_p_offloads":0,"cached_blocks":1376,"cache_hit":6144,"new_prefill":5592,"score_linear":311315.0,"score_lmetric":59283}],"chosen_score_linear":-11264.0,"chosen_score_lmetric":-472,"t_first_token":348974.751539836,"t_first_token_unix":1779779839.7654834,"t_done":348974.879649324,"t_done_unix":1779779839.8935928},{"request_id":"1362265:10:1427606:850","session_id":"1362265","input_length":12596,"t_proxy_recv":348972.533725035,"t_decision_unix":1779779837.5476658,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":11776,"affinity_cache_ratio":0.934899968243887,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":null,"tie_break_used":false,"cache_hit":11776,"estimated_new_tokens":820,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":0,"new_prefill":12596,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1881,"cache_hit":0,"new_prefill":12596,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":134193,"ongoing_decode_tokens":126198,"pending_prefill_tokens":7995,"num_requests":1,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":10752,"new_prefill":1844,"score_linear":123441.0,"score_lmetric":9839},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2144,"cache_hit":0,"new_prefill":12596,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199637,"ongoing_decode_tokens":199637,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":2627,"cache_hit":0,"new_prefill":12596,"score_linear":199637.0,"score_lmetric":25192},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":216753,"ongoing_decode_tokens":216753,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1656,"cache_hit":0,"new_prefill":12596,"score_linear":216753.0,"score_lmetric":37788},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":26923,"ongoing_decode_tokens":26923,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1556,"cache_hit":11776,"new_prefill":820,"score_linear":15147.0,"score_lmetric":820},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":317459,"ongoing_decode_tokens":124090,"pending_prefill_tokens":14169,"num_requests":3,"active_p_offloads":0,"cached_blocks":1376,"cache_hit":0,"new_prefill":12596,"score_linear":317459.0,"score_lmetric":80295}],"chosen_score_linear":15147.0,"chosen_score_lmetric":820,"t_first_token":348972.672489018,"t_first_token_unix":1779779837.6864326,"t_done":348974.926794819,"t_done_unix":1779779839.940738},{"request_id":"1370129:5:1390477:708","session_id":"1370129","input_length":87687,"t_proxy_recv":348965.761657989,"t_decision_unix":1779779830.7755992,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":85504,"affinity_cache_ratio":0.9751046335260642,"affinity_num_requests":1,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":85504,"estimated_new_tokens":2183,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":7680,"new_prefill":80007,"score_linear":-7680.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1881,"cache_hit":0,"new_prefill":87687,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":201290,"ongoing_decode_tokens":161501,"pending_prefill_tokens":1901,"num_requests":2,"active_p_offloads":0,"cached_blocks":2219,"cache_hit":0,"new_prefill":87687,"score_linear":201290.0,"score_lmetric":179176},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2142,"cache_hit":0,"new_prefill":87687,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":111950,"ongoing_decode_tokens":111950,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2627,"cache_hit":85504,"new_prefill":2183,"score_linear":26446.0,"score_lmetric":2183},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":174411,"ongoing_decode_tokens":174411,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1656,"cache_hit":0,"new_prefill":87687,"score_linear":174411.0,"score_lmetric":175374},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1556,"cache_hit":7680,"new_prefill":80007,"score_linear":-7680.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":323608,"ongoing_decode_tokens":130239,"pending_prefill_tokens":14169,"num_requests":4,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":7680,"new_prefill":80007,"score_linear":315928.0,"score_lmetric":376704}],"chosen_score_linear":26446.0,"chosen_score_lmetric":2183,"t_first_token":348967.008007615,"t_first_token_unix":1779779832.0219514,"t_done":348976.916188329,"t_done_unix":1779779841.9301314},{"request_id":"1277428:8:1426669:846","session_id":"1277428","input_length":42342,"t_proxy_recv":348969.506585864,"t_decision_unix":1779779834.5205271,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":40448,"affinity_cache_ratio":0.9552690000472345,"affinity_num_requests":2,"avg_num_requests":1.125,"fallback_score":null,"tie_break_used":false,"cache_hit":40448,"estimated_new_tokens":1894,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":0,"new_prefill":42342,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1881,"cache_hit":4608,"new_prefill":37734,"score_linear":-4608.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":126198,"ongoing_decode_tokens":126198,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2222,"cache_hit":0,"new_prefill":42342,"score_linear":126198.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2142,"cache_hit":17408,"new_prefill":24934,"score_linear":-17408.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":199637,"ongoing_decode_tokens":199637,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":2627,"cache_hit":0,"new_prefill":42342,"score_linear":199637.0,"score_lmetric":84684},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":174411,"ongoing_decode_tokens":174411,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1656,"cache_hit":40448,"new_prefill":1894,"score_linear":133963.0,"score_lmetric":3788},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":26923,"ongoing_decode_tokens":0,"pending_prefill_tokens":1835,"num_requests":1,"active_p_offloads":0,"cached_blocks":1556,"cache_hit":0,"new_prefill":42342,"score_linear":26923.0,"score_lmetric":44177},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":323608,"ongoing_decode_tokens":130239,"pending_prefill_tokens":14169,"num_requests":4,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":0,"new_prefill":42342,"score_linear":323608.0,"score_lmetric":226044}],"chosen_score_linear":133963.0,"chosen_score_lmetric":3788,"t_first_token":348970.160576968,"t_first_token_unix":1779779835.1745207,"t_done":348981.506467417,"t_done_unix":1779779846.5204103},{"request_id":"1314357:6:1422517:839","session_id":"1314357","input_length":105709,"t_proxy_recv":348974.360673353,"t_decision_unix":1779779839.374615,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":105472,"affinity_cache_ratio":0.9977579960079085,"affinity_num_requests":2,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":105472,"estimated_new_tokens":237,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":0,"new_prefill":105709,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1881,"cache_hit":0,"new_prefill":105709,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":11381,"ongoing_decode_tokens":11381,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2239,"cache_hit":0,"new_prefill":105709,"score_linear":11381.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2144,"cache_hit":0,"new_prefill":105709,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":204638,"ongoing_decode_tokens":131450,"pending_prefill_tokens":4580,"num_requests":2,"active_p_offloads":0,"cached_blocks":2634,"cache_hit":56832,"new_prefill":48877,"score_linear":147806.0,"score_lmetric":106914},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":114950,"ongoing_decode_tokens":114950,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1712,"cache_hit":105472,"new_prefill":237,"score_linear":9478.0,"score_lmetric":474},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":12596,"ongoing_decode_tokens":12596,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1560,"cache_hit":0,"new_prefill":105709,"score_linear":12596.0,"score_lmetric":105709},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":317459,"ongoing_decode_tokens":124090,"pending_prefill_tokens":14169,"num_requests":3,"active_p_offloads":0,"cached_blocks":1376,"cache_hit":0,"new_prefill":105709,"score_linear":317459.0,"score_lmetric":359634}],"chosen_score_linear":9478.0,"chosen_score_lmetric":474,"t_first_token":348974.555134942,"t_first_token_unix":1779779839.5690782,"t_done":348984.012873147,"t_done_unix":1779779849.0268168},{"request_id":"1313181:8:1329951:394","session_id":"1313181","input_length":124090,"t_proxy_recv":348891.544489475,"t_decision_unix":1779779756.558431,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":6,"chosen_idx":7,"affinity_cache_hit":61952,"affinity_cache_ratio":0.499250543960029,"affinity_num_requests":1,"avg_num_requests":1.25,"fallback_score":0,"tie_break_used":false,"cache_hit":12800,"estimated_new_tokens":111290,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":21684,"ongoing_decode_tokens":0,"pending_prefill_tokens":21684,"num_requests":1,"active_p_offloads":0,"cached_blocks":1554,"cache_hit":24064,"new_prefill":100026,"score_linear":-2380.0,"score_lmetric":121710},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":137188,"ongoing_decode_tokens":0,"pending_prefill_tokens":95204,"num_requests":2,"active_p_offloads":0,"cached_blocks":1620,"cache_hit":0,"new_prefill":124090,"score_linear":137188.0,"score_lmetric":438588},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2083,"cache_hit":0,"new_prefill":124090,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2010,"cache_hit":0,"new_prefill":124090,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":302125,"ongoing_decode_tokens":113401,"pending_prefill_tokens":5428,"num_requests":3,"active_p_offloads":0,"cached_blocks":2564,"cache_hit":0,"new_prefill":124090,"score_linear":302125.0,"score_lmetric":388554},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":203233,"ongoing_decode_tokens":135124,"pending_prefill_tokens":17421,"num_requests":3,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":124090,"score_linear":203233.0,"score_lmetric":424533},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":24442,"ongoing_decode_tokens":24442,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1385,"cache_hit":61952,"new_prefill":62138,"score_linear":-37510.0,"score_lmetric":62138},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":12800,"new_prefill":111290,"score_linear":-12800.0,"score_lmetric":0}],"chosen_score_linear":-12800.0,"chosen_score_lmetric":0,"t_first_token":348939.664453337,"t_first_token_unix":1779779804.6783972,"t_done":348987.629366842,"t_done_unix":1779779852.6433141},{"request_id":"1363093:5:1407810:790","session_id":"1363093","input_length":73188,"t_proxy_recv":348974.151936248,"t_decision_unix":1779779839.1658778,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":68608,"affinity_cache_ratio":0.9374214352079576,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":68608,"estimated_new_tokens":4580,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":0,"new_prefill":73188,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1881,"cache_hit":0,"new_prefill":73188,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":11381,"ongoing_decode_tokens":0,"pending_prefill_tokens":3189,"num_requests":0,"active_p_offloads":0,"cached_blocks":2239,"cache_hit":0,"new_prefill":73188,"score_linear":11381.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2144,"cache_hit":0,"new_prefill":73188,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":131450,"ongoing_decode_tokens":131450,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2634,"cache_hit":68608,"new_prefill":4580,"score_linear":62842.0,"score_lmetric":4580},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":216753,"ongoing_decode_tokens":216753,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1656,"cache_hit":0,"new_prefill":73188,"score_linear":216753.0,"score_lmetric":219564},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":12596,"ongoing_decode_tokens":12596,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1560,"cache_hit":0,"new_prefill":73188,"score_linear":12596.0,"score_lmetric":73188},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":317459,"ongoing_decode_tokens":124090,"pending_prefill_tokens":14169,"num_requests":3,"active_p_offloads":0,"cached_blocks":1376,"cache_hit":0,"new_prefill":73188,"score_linear":317459.0,"score_lmetric":262071}],"chosen_score_linear":62842.0,"chosen_score_lmetric":4580,"t_first_token":348976.282178839,"t_first_token_unix":1779779841.2961226,"t_done":348988.187905366,"t_done_unix":1779779853.201849},{"request_id":"1314357:7:1427957:851","session_id":"1314357","input_length":106279,"t_proxy_recv":348984.031135851,"t_decision_unix":1779779849.0450776,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":105984,"affinity_cache_ratio":0.9972242870181315,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":105984,"estimated_new_tokens":295,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":0,"new_prefill":106279,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1881,"cache_hit":0,"new_prefill":106279,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":2245,"cache_hit":0,"new_prefill":106279,"score_linear":0.0,"score_lmetric":-106279},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":1840,"ongoing_decode_tokens":1840,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2144,"cache_hit":0,"new_prefill":106279,"score_linear":1840.0,"score_lmetric":106279},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":223094,"ongoing_decode_tokens":116951,"pending_prefill_tokens":18079,"num_requests":2,"active_p_offloads":0,"cached_blocks":2639,"cache_hit":56832,"new_prefill":49447,"score_linear":166262.0,"score_lmetric":135052},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":72608,"ongoing_decode_tokens":72608,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1717,"cache_hit":105984,"new_prefill":295,"score_linear":-33376.0,"score_lmetric":295},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1562,"cache_hit":0,"new_prefill":106279,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":317459,"ongoing_decode_tokens":124090,"pending_prefill_tokens":14169,"num_requests":3,"active_p_offloads":0,"cached_blocks":1376,"cache_hit":0,"new_prefill":106279,"score_linear":317459.0,"score_lmetric":361344}],"chosen_score_linear":-33376.0,"chosen_score_lmetric":295,"t_first_token":348984.217386371,"t_first_token_unix":1779779849.2313302,"t_done":348991.641568946,"t_done_unix":1779779856.6555126},{"request_id":"1381387:7:1411928:819","session_id":"1381387","input_length":72608,"t_proxy_recv":348917.358056537,"t_decision_unix":1779779782.3719974,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":71680,"affinity_cache_ratio":0.9872190392243279,"affinity_num_requests":2,"avg_num_requests":2.0,"fallback_score":null,"tie_break_used":false,"cache_hit":71680,"estimated_new_tokens":928,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":154908,"ongoing_decode_tokens":21684,"pending_prefill_tokens":12392,"num_requests":3,"active_p_offloads":0,"cached_blocks":1554,"cache_hit":0,"new_prefill":72608,"score_linear":154908.0,"score_lmetric":255000},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1828,"cache_hit":0,"new_prefill":72608,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":283010,"ongoing_decode_tokens":60212,"pending_prefill_tokens":16462,"num_requests":4,"active_p_offloads":0,"cached_blocks":2083,"cache_hit":0,"new_prefill":72608,"score_linear":283010.0,"score_lmetric":356280},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":107292,"ongoing_decode_tokens":0,"pending_prefill_tokens":14108,"num_requests":1,"active_p_offloads":0,"cached_blocks":2101,"cache_hit":512,"new_prefill":72096,"score_linear":106780.0,"score_lmetric":86204},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":253194,"ongoing_decode_tokens":124993,"pending_prefill_tokens":1225,"num_requests":2,"active_p_offloads":0,"cached_blocks":2566,"cache_hit":0,"new_prefill":72608,"score_linear":253194.0,"score_lmetric":147666},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":135124,"ongoing_decode_tokens":135124,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1640,"cache_hit":71680,"new_prefill":928,"score_linear":63444.0,"score_lmetric":1856},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1472,"cache_hit":0,"new_prefill":72608,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":323608,"ongoing_decode_tokens":0,"pending_prefill_tokens":127000,"num_requests":4,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":0,"new_prefill":72608,"score_linear":323608.0,"score_lmetric":798432}],"chosen_score_linear":63444.0,"chosen_score_lmetric":1856,"t_first_token":348917.999314002,"t_first_token_unix":1779779783.013258,"t_done":348992.909683586,"t_done_unix":1779779857.923627},{"request_id":"1270340:5:1417075:829","session_id":"1270340","input_length":43763,"t_proxy_recv":348935.606872229,"t_decision_unix":1779779800.6208138,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":40960,"affinity_cache_ratio":0.9359504604346137,"affinity_num_requests":1,"avg_num_requests":1.875,"fallback_score":null,"tie_break_used":false,"cache_hit":40960,"estimated_new_tokens":2803,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":154908,"ongoing_decode_tokens":134942,"pending_prefill_tokens":9214,"num_requests":3,"active_p_offloads":0,"cached_blocks":1554,"cache_hit":0,"new_prefill":43763,"score_linear":154908.0,"score_lmetric":158931},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1871,"cache_hit":0,"new_prefill":43763,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":232914,"ongoing_decode_tokens":115168,"pending_prefill_tokens":8178,"num_requests":2,"active_p_offloads":0,"cached_blocks":2150,"cache_hit":0,"new_prefill":43763,"score_linear":232914.0,"score_lmetric":103882},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":147622,"ongoing_decode_tokens":0,"pending_prefill_tokens":21158,"num_requests":2,"active_p_offloads":0,"cached_blocks":2101,"cache_hit":1536,"new_prefill":42227,"score_linear":146086.0,"score_lmetric":126770},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":124993,"ongoing_decode_tokens":124993,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2568,"cache_hit":40960,"new_prefill":2803,"score_linear":84033.0,"score_lmetric":2803},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":207732,"ongoing_decode_tokens":207732,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1644,"cache_hit":0,"new_prefill":43763,"score_linear":207732.0,"score_lmetric":131289},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1503,"cache_hit":0,"new_prefill":43763,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":323608,"ongoing_decode_tokens":0,"pending_prefill_tokens":127000,"num_requests":4,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":0,"new_prefill":43763,"score_linear":323608.0,"score_lmetric":683052}],"chosen_score_linear":84033.0,"chosen_score_lmetric":2803,"t_first_token":348943.148515258,"t_first_token_unix":1779779808.1624591,"t_done":348993.232953248,"t_done_unix":1779779858.2468967},{"request_id":"1314357:8:1431749:854","session_id":"1314357","input_length":106805,"t_proxy_recv":348991.657234776,"t_decision_unix":1779779856.6711762,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":106496,"affinity_cache_ratio":0.9971068770188661,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":106496,"estimated_new_tokens":309,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":125947,"ongoing_decode_tokens":0,"pending_prefill_tokens":11771,"num_requests":1,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":0,"new_prefill":106805,"score_linear":125947.0,"score_lmetric":118576},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1881,"cache_hit":0,"new_prefill":106805,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":2245,"cache_hit":0,"new_prefill":106805,"score_linear":0.0,"score_lmetric":-106805},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":1840,"ongoing_decode_tokens":1840,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2144,"cache_hit":0,"new_prefill":106805,"score_linear":1840.0,"score_lmetric":106805},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":149906,"ongoing_decode_tokens":149906,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2648,"cache_hit":56832,"new_prefill":49973,"score_linear":93074.0,"score_lmetric":49973},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":72608,"ongoing_decode_tokens":72608,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1718,"cache_hit":106496,"new_prefill":309,"score_linear":-33888.0,"score_lmetric":309},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1562,"cache_hit":0,"new_prefill":106805,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":255813,"ongoing_decode_tokens":119977,"pending_prefill_tokens":10908,"num_requests":3,"active_p_offloads":0,"cached_blocks":1592,"cache_hit":0,"new_prefill":106805,"score_linear":255813.0,"score_lmetric":353139}],"chosen_score_linear":-33888.0,"chosen_score_lmetric":309,"t_first_token":348991.833679585,"t_first_token_unix":1779779856.847623,"t_done":348993.431241863,"t_done_unix":1779779858.4451852},{"request_id":"1399948:9:1433544:857","session_id":"1399948","input_length":11993,"t_proxy_recv":348993.722239311,"t_decision_unix":1779779858.7361805,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":11264,"affinity_cache_ratio":0.9392145418160593,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":11264,"estimated_new_tokens":729,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":125947,"ongoing_decode_tokens":0,"pending_prefill_tokens":11771,"num_requests":1,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":0,"new_prefill":11993,"score_linear":125947.0,"score_lmetric":23764},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1881,"cache_hit":0,"new_prefill":11993,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":40567,"ongoing_decode_tokens":0,"pending_prefill_tokens":1655,"num_requests":0,"active_p_offloads":0,"cached_blocks":2245,"cache_hit":11264,"new_prefill":729,"score_linear":29303.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":75718,"ongoing_decode_tokens":1840,"pending_prefill_tokens":2710,"num_requests":2,"active_p_offloads":0,"cached_blocks":2144,"cache_hit":0,"new_prefill":11993,"score_linear":75718.0,"score_lmetric":29406},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2655,"cache_hit":0,"new_prefill":11993,"score_linear":106143.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1721,"cache_hit":0,"new_prefill":11993,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1562,"cache_hit":0,"new_prefill":11993,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":255813,"ongoing_decode_tokens":119977,"pending_prefill_tokens":10908,"num_requests":3,"active_p_offloads":0,"cached_blocks":1592,"cache_hit":6144,"new_prefill":5849,"score_linear":249669.0,"score_lmetric":50271}],"chosen_score_linear":29303.0,"chosen_score_lmetric":0,"t_first_token":348993.956459646,"t_first_token_unix":1779779858.970404,"t_done":348994.052693884,"t_done_unix":1779779859.0666373},{"request_id":"1382089:3:1433211:856","session_id":"1382089","input_length":40567,"t_proxy_recv":348992.503477758,"t_decision_unix":1779779857.5174189,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":38912,"affinity_cache_ratio":0.9592032933172283,"affinity_num_requests":-1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":38912,"estimated_new_tokens":1655,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":125947,"ongoing_decode_tokens":0,"pending_prefill_tokens":11771,"num_requests":1,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":0,"new_prefill":40567,"score_linear":125947.0,"score_lmetric":52338},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1881,"cache_hit":0,"new_prefill":40567,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":2245,"cache_hit":38912,"new_prefill":1655,"score_linear":-38912.0,"score_lmetric":-1655},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":75718,"ongoing_decode_tokens":1840,"pending_prefill_tokens":2710,"num_requests":2,"active_p_offloads":0,"cached_blocks":2144,"cache_hit":0,"new_prefill":40567,"score_linear":75718.0,"score_lmetric":86554},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":149906,"ongoing_decode_tokens":149906,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2648,"cache_hit":0,"new_prefill":40567,"score_linear":149906.0,"score_lmetric":40567},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":179413,"ongoing_decode_tokens":179413,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1718,"cache_hit":0,"new_prefill":40567,"score_linear":179413.0,"score_lmetric":81134},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1562,"cache_hit":0,"new_prefill":40567,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":255813,"ongoing_decode_tokens":119977,"pending_prefill_tokens":10908,"num_requests":3,"active_p_offloads":0,"cached_blocks":1592,"cache_hit":0,"new_prefill":40567,"score_linear":255813.0,"score_lmetric":154425}],"chosen_score_linear":-38912.0,"chosen_score_lmetric":-1655,"t_first_token":348993.91482229,"t_first_token_unix":1779779858.9287658,"t_done":348996.101151038,"t_done_unix":1779779861.1150942},{"request_id":"1399948:10:1434418:858","session_id":"1399948","input_length":12032,"t_proxy_recv":348996.735335116,"t_decision_unix":1779779861.749276,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":11776,"affinity_cache_ratio":0.9787234042553191,"affinity_num_requests":-1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":11776,"estimated_new_tokens":256,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":125947,"ongoing_decode_tokens":125947,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":0,"new_prefill":12032,"score_linear":125947.0,"score_lmetric":12032},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1881,"cache_hit":0,"new_prefill":12032,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":2249,"cache_hit":11776,"new_prefill":256,"score_linear":-11776.0,"score_lmetric":-256},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":75718,"ongoing_decode_tokens":1840,"pending_prefill_tokens":2710,"num_requests":2,"active_p_offloads":0,"cached_blocks":2144,"cache_hit":0,"new_prefill":12032,"score_linear":75718.0,"score_lmetric":29484},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2655,"cache_hit":0,"new_prefill":12032,"score_linear":106143.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1721,"cache_hit":0,"new_prefill":12032,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1562,"cache_hit":0,"new_prefill":12032,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":255813,"ongoing_decode_tokens":119977,"pending_prefill_tokens":10908,"num_requests":3,"active_p_offloads":0,"cached_blocks":1592,"cache_hit":6144,"new_prefill":5888,"score_linear":249669.0,"score_lmetric":50388}],"chosen_score_linear":-11776.0,"chosen_score_lmetric":-256,"t_first_token":348996.770423342,"t_first_token_unix":1779779861.7843673,"t_done":348996.983632413,"t_done_unix":1779779861.9975762},{"request_id":"1399948:11:1435163:860","session_id":"1399948","input_length":12106,"t_proxy_recv":348999.436929955,"t_decision_unix":1779779864.450871,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":11776,"affinity_cache_ratio":0.9727407896910623,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":11776,"estimated_new_tokens":330,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":125947,"ongoing_decode_tokens":125947,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":0,"new_prefill":12106,"score_linear":125947.0,"score_lmetric":12106},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1881,"cache_hit":0,"new_prefill":12106,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2249,"cache_hit":11776,"new_prefill":330,"score_linear":-11776.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":75718,"ongoing_decode_tokens":1840,"pending_prefill_tokens":2710,"num_requests":2,"active_p_offloads":0,"cached_blocks":2144,"cache_hit":0,"new_prefill":12106,"score_linear":75718.0,"score_lmetric":29632},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":151412,"ongoing_decode_tokens":151412,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2655,"cache_hit":0,"new_prefill":12106,"score_linear":151412.0,"score_lmetric":12106},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1721,"cache_hit":0,"new_prefill":12106,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1562,"cache_hit":0,"new_prefill":12106,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":255813,"ongoing_decode_tokens":119977,"pending_prefill_tokens":10908,"num_requests":3,"active_p_offloads":0,"cached_blocks":1592,"cache_hit":6144,"new_prefill":5962,"score_linear":249669.0,"score_lmetric":50610}],"chosen_score_linear":-11776.0,"chosen_score_lmetric":0,"t_first_token":348999.474852764,"t_first_token_unix":1779779864.488796,"t_done":348999.523328536,"t_done_unix":1779779864.5372715},{"request_id":"1270340:6:1434904:859","session_id":"1270340","input_length":45269,"t_proxy_recv":348998.376769828,"t_decision_unix":1779779863.390711,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":45056,"affinity_cache_ratio":0.9952947933464402,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":45056,"estimated_new_tokens":213,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":125947,"ongoing_decode_tokens":125947,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":0,"new_prefill":45269,"score_linear":125947.0,"score_lmetric":45269},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1881,"cache_hit":0,"new_prefill":45269,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2249,"cache_hit":0,"new_prefill":45269,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":75718,"ongoing_decode_tokens":1840,"pending_prefill_tokens":2710,"num_requests":2,"active_p_offloads":0,"cached_blocks":2144,"cache_hit":1536,"new_prefill":43733,"score_linear":74182.0,"score_lmetric":92886},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2655,"cache_hit":45056,"new_prefill":213,"score_linear":61087.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1721,"cache_hit":0,"new_prefill":45269,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1562,"cache_hit":0,"new_prefill":45269,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":255813,"ongoing_decode_tokens":119977,"pending_prefill_tokens":10908,"num_requests":3,"active_p_offloads":0,"cached_blocks":1592,"cache_hit":0,"new_prefill":45269,"score_linear":255813.0,"score_lmetric":168531}],"chosen_score_linear":61087.0,"chosen_score_lmetric":0,"t_first_token":348998.475172841,"t_first_token_unix":1779779863.4891162,"t_done":349001.228840533,"t_done_unix":1779779866.2427838},{"request_id":"1275274:2:1405679:780","session_id":"1275274","input_length":119977,"t_proxy_recv":348895.53857813,"t_decision_unix":1779779760.5525198,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":116224,"affinity_cache_ratio":0.9687190044758579,"affinity_num_requests":2,"avg_num_requests":2.25,"fallback_score":null,"tie_break_used":false,"cache_hit":116224,"estimated_new_tokens":3753,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":154908,"ongoing_decode_tokens":21684,"pending_prefill_tokens":12392,"num_requests":3,"active_p_offloads":0,"cached_blocks":1554,"cache_hit":0,"new_prefill":119977,"score_linear":154908.0,"score_lmetric":397107},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":137188,"ongoing_decode_tokens":0,"pending_prefill_tokens":95204,"num_requests":2,"active_p_offloads":0,"cached_blocks":1620,"cache_hit":0,"new_prefill":119977,"score_linear":137188.0,"score_lmetric":430362},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":145650,"ongoing_decode_tokens":21009,"pending_prefill_tokens":5345,"num_requests":3,"active_p_offloads":0,"cached_blocks":2083,"cache_hit":0,"new_prefill":119977,"score_linear":145650.0,"score_lmetric":375966},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":32944,"ongoing_decode_tokens":0,"pending_prefill_tokens":32944,"num_requests":1,"active_p_offloads":0,"cached_blocks":2010,"cache_hit":0,"new_prefill":119977,"score_linear":32944.0,"score_lmetric":152921},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":302125,"ongoing_decode_tokens":113401,"pending_prefill_tokens":5428,"num_requests":3,"active_p_offloads":0,"cached_blocks":2564,"cache_hit":0,"new_prefill":119977,"score_linear":302125.0,"score_lmetric":376215},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":203233,"ongoing_decode_tokens":203233,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":119977,"score_linear":203233.0,"score_lmetric":359931},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":24442,"ongoing_decode_tokens":24442,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1407,"cache_hit":0,"new_prefill":119977,"score_linear":24442.0,"score_lmetric":119977},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":130239,"ongoing_decode_tokens":0,"pending_prefill_tokens":112831,"num_requests":2,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":116224,"new_prefill":3753,"score_linear":14015.0,"score_lmetric":233168}],"chosen_score_linear":14015.0,"chosen_score_lmetric":233168,"t_first_token":348986.630052655,"t_first_token_unix":1779779851.6439965,"t_done":349002.276671824,"t_done_unix":1779779867.2906156},{"request_id":"1314357:9:1435219:861","session_id":"1314357","input_length":106959,"t_proxy_recv":348999.674858845,"t_decision_unix":1779779864.6888,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":106496,"affinity_cache_ratio":0.9956712385119532,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":106496,"estimated_new_tokens":463,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":125947,"ongoing_decode_tokens":125947,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":0,"new_prefill":106959,"score_linear":125947.0,"score_lmetric":106959},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1881,"cache_hit":0,"new_prefill":106959,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2249,"cache_hit":0,"new_prefill":106959,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":75718,"ongoing_decode_tokens":1840,"pending_prefill_tokens":2710,"num_requests":2,"active_p_offloads":0,"cached_blocks":2144,"cache_hit":0,"new_prefill":106959,"score_linear":75718.0,"score_lmetric":219338},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":151412,"ongoing_decode_tokens":151412,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2655,"cache_hit":56832,"new_prefill":50127,"score_linear":94580.0,"score_lmetric":50127},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1721,"cache_hit":106496,"new_prefill":463,"score_linear":-106496.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1562,"cache_hit":0,"new_prefill":106959,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":255813,"ongoing_decode_tokens":119977,"pending_prefill_tokens":10908,"num_requests":3,"active_p_offloads":0,"cached_blocks":1592,"cache_hit":0,"new_prefill":106959,"score_linear":255813.0,"score_lmetric":353601}],"chosen_score_linear":-106496.0,"chosen_score_lmetric":0,"t_first_token":348999.837750874,"t_first_token_unix":1779779864.851694,"t_done":349002.435388482,"t_done_unix":1779779867.449332},{"request_id":"1313181:8:1385690:686","session_id":"1313181","input_length":62444,"t_proxy_recv":348987.641272462,"t_decision_unix":1779779852.655214,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":61952,"affinity_cache_ratio":0.9921209403625648,"affinity_num_requests":2,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":61952,"estimated_new_tokens":492,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":125947,"ongoing_decode_tokens":0,"pending_prefill_tokens":11771,"num_requests":1,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":24064,"new_prefill":38380,"score_linear":101883.0,"score_lmetric":50151},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1881,"cache_hit":0,"new_prefill":62444,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":2245,"cache_hit":0,"new_prefill":62444,"score_linear":0.0,"score_lmetric":-62444},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":1840,"ongoing_decode_tokens":1840,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2144,"cache_hit":0,"new_prefill":62444,"score_linear":1840.0,"score_lmetric":62444},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":223094,"ongoing_decode_tokens":116951,"pending_prefill_tokens":18079,"num_requests":2,"active_p_offloads":0,"cached_blocks":2639,"cache_hit":0,"new_prefill":62444,"score_linear":223094.0,"score_lmetric":161046},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":178887,"ongoing_decode_tokens":178887,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1717,"cache_hit":0,"new_prefill":62444,"score_linear":178887.0,"score_lmetric":124888},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1562,"cache_hit":61952,"new_prefill":492,"score_linear":-61952.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":193369,"ongoing_decode_tokens":119977,"pending_prefill_tokens":10416,"num_requests":2,"active_p_offloads":0,"cached_blocks":1592,"cache_hit":61952,"new_prefill":492,"score_linear":131417.0,"score_lmetric":21816}],"chosen_score_linear":131417.0,"chosen_score_lmetric":21816,"t_first_token":349000.801938253,"t_first_token_unix":1779779865.8158815,"t_done":349002.46221357,"t_done_unix":1779779867.476157},{"request_id":"1399948:12:1436179:863","session_id":"1399948","input_length":12164,"t_proxy_recv":349002.863160406,"t_decision_unix":1779779867.877102,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":11776,"affinity_cache_ratio":0.9681025978296613,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":11776,"estimated_new_tokens":388,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":155359,"ongoing_decode_tokens":125947,"pending_prefill_tokens":6884,"num_requests":2,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":0,"new_prefill":12164,"score_linear":155359.0,"score_lmetric":38096},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1881,"cache_hit":0,"new_prefill":12164,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2249,"cache_hit":11776,"new_prefill":388,"score_linear":-11776.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":75718,"ongoing_decode_tokens":1840,"pending_prefill_tokens":2710,"num_requests":2,"active_p_offloads":0,"cached_blocks":2144,"cache_hit":0,"new_prefill":12164,"score_linear":75718.0,"score_lmetric":29748},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2655,"cache_hit":0,"new_prefill":12164,"score_linear":106143.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1722,"cache_hit":0,"new_prefill":12164,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":129673,"ongoing_decode_tokens":0,"pending_prefill_tokens":67721,"num_requests":1,"active_p_offloads":0,"cached_blocks":1562,"cache_hit":0,"new_prefill":12164,"score_linear":129673.0,"score_lmetric":79885},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":73392,"ongoing_decode_tokens":73392,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1600,"cache_hit":6144,"new_prefill":6020,"score_linear":67248.0,"score_lmetric":6020}],"chosen_score_linear":-11776.0,"chosen_score_lmetric":0,"t_first_token":349002.909214844,"t_first_token_unix":1779779867.923158,"t_done":349003.042243648,"t_done_unix":1779779868.0561867},{"request_id":"1364090:6:1408777:801","session_id":"1364090","input_length":73392,"t_proxy_recv":348906.368355662,"t_decision_unix":1779779771.3822973,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":62976,"affinity_cache_ratio":0.8580771746239372,"affinity_num_requests":3,"avg_num_requests":2.5,"fallback_score":null,"tie_break_used":false,"cache_hit":62976,"estimated_new_tokens":10416,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":154908,"ongoing_decode_tokens":21684,"pending_prefill_tokens":12392,"num_requests":3,"active_p_offloads":0,"cached_blocks":1554,"cache_hit":7680,"new_prefill":65712,"score_linear":147228.0,"score_lmetric":234312},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":1801,"ongoing_decode_tokens":0,"pending_prefill_tokens":1801,"num_requests":1,"active_p_offloads":0,"cached_blocks":1806,"cache_hit":0,"new_prefill":73392,"score_linear":1801.0,"score_lmetric":75193},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":283010,"ongoing_decode_tokens":60212,"pending_prefill_tokens":16462,"num_requests":5,"active_p_offloads":0,"cached_blocks":2083,"cache_hit":0,"new_prefill":73392,"score_linear":283010.0,"score_lmetric":449270},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":6377,"ongoing_decode_tokens":6377,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2089,"cache_hit":0,"new_prefill":73392,"score_linear":6377.0,"score_lmetric":73392},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":303466,"ongoing_decode_tokens":124993,"pending_prefill_tokens":1833,"num_requests":3,"active_p_offloads":0,"cached_blocks":2565,"cache_hit":7680,"new_prefill":65712,"score_linear":295786.0,"score_lmetric":202635},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":162800,"ongoing_decode_tokens":162800,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1625,"cache_hit":0,"new_prefill":73392,"score_linear":162800.0,"score_lmetric":220176},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":11070,"ongoing_decode_tokens":0,"pending_prefill_tokens":11070,"num_requests":1,"active_p_offloads":0,"cached_blocks":1439,"cache_hit":7680,"new_prefill":65712,"score_linear":3390.0,"score_lmetric":76782},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":250216,"ongoing_decode_tokens":0,"pending_prefill_tokens":116584,"num_requests":3,"active_p_offloads":0,"cached_blocks":1373,"cache_hit":62976,"new_prefill":10416,"score_linear":187240.0,"score_lmetric":381000}],"chosen_score_linear":187240.0,"chosen_score_lmetric":381000,"t_first_token":349000.798521607,"t_first_token_unix":1779779865.8124664,"t_done":349005.660467381,"t_done_unix":1779779870.6744108},{"request_id":"1342921:5:1433192:855","session_id":"1342921","input_length":73878,"t_proxy_recv":348992.431547906,"t_decision_unix":1779779857.4454892,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":71168,"affinity_cache_ratio":0.9633179024878854,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":71168,"estimated_new_tokens":2710,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":125947,"ongoing_decode_tokens":0,"pending_prefill_tokens":11771,"num_requests":1,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":0,"new_prefill":73878,"score_linear":125947.0,"score_lmetric":85649},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1881,"cache_hit":0,"new_prefill":73878,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":2245,"cache_hit":0,"new_prefill":73878,"score_linear":0.0,"score_lmetric":-73878},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":1840,"ongoing_decode_tokens":1840,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2144,"cache_hit":71168,"new_prefill":2710,"score_linear":-69328.0,"score_lmetric":2710},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":149906,"ongoing_decode_tokens":149906,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2648,"cache_hit":0,"new_prefill":73878,"score_linear":149906.0,"score_lmetric":73878},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":179413,"ongoing_decode_tokens":179413,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1718,"cache_hit":0,"new_prefill":73878,"score_linear":179413.0,"score_lmetric":147756},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1562,"cache_hit":0,"new_prefill":73878,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":255813,"ongoing_decode_tokens":119977,"pending_prefill_tokens":10908,"num_requests":3,"active_p_offloads":0,"cached_blocks":1592,"cache_hit":0,"new_prefill":73878,"score_linear":255813.0,"score_lmetric":254358}],"chosen_score_linear":-69328.0,"chosen_score_lmetric":2710,"t_first_token":349010.9528817,"t_first_token_unix":1779779875.9668255,"t_done":349012.217072385,"t_done_unix":1779779877.231016},{"request_id":"1407599:2:1430593:852","session_id":"1407599","input_length":1840,"t_proxy_recv":348983.258142611,"t_decision_unix":1779779848.2720838,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":1536,"affinity_cache_ratio":0.8347826086956521,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":1536,"estimated_new_tokens":304,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":0,"new_prefill":1840,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1881,"cache_hit":0,"new_prefill":1840,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":2245,"cache_hit":0,"new_prefill":1840,"score_linear":0.0,"score_lmetric":-1840},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2144,"cache_hit":1536,"new_prefill":304,"score_linear":-1536.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":223094,"ongoing_decode_tokens":116951,"pending_prefill_tokens":18079,"num_requests":2,"active_p_offloads":0,"cached_blocks":2639,"cache_hit":0,"new_prefill":1840,"score_linear":223094.0,"score_lmetric":39838},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":178317,"ongoing_decode_tokens":178317,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1716,"cache_hit":0,"new_prefill":1840,"score_linear":178317.0,"score_lmetric":3680},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1562,"cache_hit":0,"new_prefill":1840,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":317459,"ongoing_decode_tokens":124090,"pending_prefill_tokens":14169,"num_requests":3,"active_p_offloads":0,"cached_blocks":1376,"cache_hit":0,"new_prefill":1840,"score_linear":317459.0,"score_lmetric":48027}],"chosen_score_linear":-1536.0,"chosen_score_lmetric":0,"t_first_token":348983.294798688,"t_first_token_unix":1779779848.3087418,"t_done":349013.192796085,"t_done_unix":1779779878.2067397},{"request_id":"1314357:10:1439373:867","session_id":"1314357","input_length":107228,"t_proxy_recv":349013.52096555,"t_decision_unix":1779779878.5349066,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":107008,"affinity_cache_ratio":0.9979482970865818,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":107008,"estimated_new_tokens":220,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":282847,"ongoing_decode_tokens":155359,"pending_prefill_tokens":5120,"num_requests":3,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":0,"new_prefill":107228,"score_linear":282847.0,"score_lmetric":337044},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":52920,"ongoing_decode_tokens":0,"pending_prefill_tokens":3768,"num_requests":1,"active_p_offloads":0,"cached_blocks":1881,"cache_hit":0,"new_prefill":107228,"score_linear":52920.0,"score_lmetric":110996},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2249,"cache_hit":0,"new_prefill":107228,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2153,"cache_hit":0,"new_prefill":107228,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2655,"cache_hit":56832,"new_prefill":50396,"score_linear":49311.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1722,"cache_hit":107008,"new_prefill":220,"score_linear":-107008.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":158384,"ongoing_decode_tokens":0,"pending_prefill_tokens":69296,"num_requests":2,"active_p_offloads":0,"cached_blocks":1562,"cache_hit":0,"new_prefill":107228,"score_linear":158384.0,"score_lmetric":353048},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":86193,"ongoing_decode_tokens":86193,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1621,"cache_hit":0,"new_prefill":107228,"score_linear":86193.0,"score_lmetric":107228}],"chosen_score_linear":-107008.0,"chosen_score_lmetric":0,"t_first_token":349013.676026305,"t_first_token_unix":1779779878.68997,"t_done":349013.971168012,"t_done_unix":1779779878.9851148},{"request_id":"1268861:10:1439325:866","session_id":"1268861","input_length":52920,"t_proxy_recv":349013.390585932,"t_decision_unix":1779779878.404527,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":49152,"affinity_cache_ratio":0.9287981859410431,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":49152,"estimated_new_tokens":3768,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":282847,"ongoing_decode_tokens":155359,"pending_prefill_tokens":5120,"num_requests":3,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":0,"new_prefill":52920,"score_linear":282847.0,"score_lmetric":174120},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1881,"cache_hit":49152,"new_prefill":3768,"score_linear":-49152.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2249,"cache_hit":0,"new_prefill":52920,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2153,"cache_hit":0,"new_prefill":52920,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2655,"cache_hit":0,"new_prefill":52920,"score_linear":106143.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1722,"cache_hit":0,"new_prefill":52920,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":158384,"ongoing_decode_tokens":0,"pending_prefill_tokens":69296,"num_requests":2,"active_p_offloads":0,"cached_blocks":1562,"cache_hit":0,"new_prefill":52920,"score_linear":158384.0,"score_lmetric":244432},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":86193,"ongoing_decode_tokens":86193,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1621,"cache_hit":12288,"new_prefill":40632,"score_linear":73905.0,"score_lmetric":40632}],"chosen_score_linear":-49152.0,"chosen_score_lmetric":0,"t_first_token":349014.61157932,"t_first_token_unix":1779779879.6255236,"t_done":349015.728899904,"t_done_unix":1779779880.7428432},{"request_id":"1364090:7:1412644:820","session_id":"1364090","input_length":86193,"t_proxy_recv":349005.679533111,"t_decision_unix":1779779870.6934745,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":73728,"affinity_cache_ratio":0.8553826876892555,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":73728,"estimated_new_tokens":12465,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":155359,"ongoing_decode_tokens":155359,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":7680,"new_prefill":78513,"score_linear":147679.0,"score_lmetric":157026},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1881,"cache_hit":0,"new_prefill":86193,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2249,"cache_hit":0,"new_prefill":86193,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":75718,"ongoing_decode_tokens":1840,"pending_prefill_tokens":2710,"num_requests":2,"active_p_offloads":0,"cached_blocks":2144,"cache_hit":0,"new_prefill":86193,"score_linear":75718.0,"score_lmetric":177806},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2655,"cache_hit":7680,"new_prefill":78513,"score_linear":98463.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1722,"cache_hit":0,"new_prefill":86193,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":129673,"ongoing_decode_tokens":0,"pending_prefill_tokens":67721,"num_requests":1,"active_p_offloads":0,"cached_blocks":1562,"cache_hit":7680,"new_prefill":78513,"score_linear":121993.0,"score_lmetric":146234},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1621,"cache_hit":73728,"new_prefill":12465,"score_linear":-73728.0,"score_lmetric":0}],"chosen_score_linear":-73728.0,"chosen_score_lmetric":0,"t_first_token":349011.935882494,"t_first_token_unix":1779779876.9498267,"t_done":349016.880160922,"t_done_unix":1779779881.8941042},{"request_id":"1371622:2:1442319:869","session_id":"1371622","input_length":19056,"t_proxy_recv":349023.238905202,"t_decision_unix":1779779888.2528465,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":13312,"affinity_cache_ratio":0.6985726280436608,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":13312,"estimated_new_tokens":5744,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":282847,"ongoing_decode_tokens":155359,"pending_prefill_tokens":5120,"num_requests":3,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":0,"new_prefill":19056,"score_linear":282847.0,"score_lmetric":72528},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":19056,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2249,"cache_hit":0,"new_prefill":19056,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2153,"cache_hit":0,"new_prefill":19056,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":169296,"ongoing_decode_tokens":106143,"pending_prefill_tokens":14001,"num_requests":1,"active_p_offloads":0,"cached_blocks":2655,"cache_hit":11264,"new_prefill":7792,"score_linear":158032.0,"score_lmetric":21793},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1722,"cache_hit":13312,"new_prefill":5744,"score_linear":-13312.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":158384,"ongoing_decode_tokens":0,"pending_prefill_tokens":69296,"num_requests":2,"active_p_offloads":0,"cached_blocks":1562,"cache_hit":0,"new_prefill":19056,"score_linear":158384.0,"score_lmetric":176704},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":96849,"ongoing_decode_tokens":96849,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1646,"cache_hit":0,"new_prefill":19056,"score_linear":96849.0,"score_lmetric":19056}],"chosen_score_linear":-13312.0,"chosen_score_lmetric":0,"t_first_token":349025.157529444,"t_first_token_unix":1779779890.1714728,"t_done":349026.922746225,"t_done_unix":1779779891.9366896},{"request_id":"1364090:8:1417347:830","session_id":"1364090","input_length":96849,"t_proxy_recv":349016.925324554,"t_decision_unix":1779779881.9392662,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":86528,"affinity_cache_ratio":0.893432043696889,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":86528,"estimated_new_tokens":10321,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":282847,"ongoing_decode_tokens":155359,"pending_prefill_tokens":5120,"num_requests":3,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":7680,"new_prefill":89169,"score_linear":275167.0,"score_lmetric":282867},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":96849,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2249,"cache_hit":0,"new_prefill":96849,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2153,"cache_hit":0,"new_prefill":96849,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2655,"cache_hit":7680,"new_prefill":89169,"score_linear":98463.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1722,"cache_hit":0,"new_prefill":96849,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":158384,"ongoing_decode_tokens":0,"pending_prefill_tokens":69296,"num_requests":2,"active_p_offloads":0,"cached_blocks":1562,"cache_hit":7680,"new_prefill":89169,"score_linear":150704.0,"score_lmetric":316930},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1646,"cache_hit":86528,"new_prefill":10321,"score_linear":-86528.0,"score_lmetric":0}],"chosen_score_linear":-86528.0,"chosen_score_lmetric":0,"t_first_token":349022.832123261,"t_first_token_unix":1779779887.846067,"t_done":349027.403310351,"t_done_unix":1779779892.4172537},{"request_id":"1314357:11:1442554:870","session_id":"1314357","input_length":108166,"t_proxy_recv":349024.037768367,"t_decision_unix":1779779889.0517094,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":107008,"affinity_cache_ratio":0.9892942329382616,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":107008,"estimated_new_tokens":1158,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":282847,"ongoing_decode_tokens":155359,"pending_prefill_tokens":5120,"num_requests":3,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":0,"new_prefill":108166,"score_linear":282847.0,"score_lmetric":339858},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":108166,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2249,"cache_hit":0,"new_prefill":108166,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2153,"cache_hit":0,"new_prefill":108166,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":169296,"ongoing_decode_tokens":106143,"pending_prefill_tokens":14001,"num_requests":1,"active_p_offloads":0,"cached_blocks":2655,"cache_hit":56832,"new_prefill":51334,"score_linear":112464.0,"score_lmetric":65335},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":19056,"ongoing_decode_tokens":0,"pending_prefill_tokens":5744,"num_requests":1,"active_p_offloads":0,"cached_blocks":1722,"cache_hit":107008,"new_prefill":1158,"score_linear":-87952.0,"score_lmetric":6902},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":158384,"ongoing_decode_tokens":0,"pending_prefill_tokens":69296,"num_requests":2,"active_p_offloads":0,"cached_blocks":1562,"cache_hit":0,"new_prefill":108166,"score_linear":158384.0,"score_lmetric":354924},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":96849,"ongoing_decode_tokens":96849,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1646,"cache_hit":0,"new_prefill":108166,"score_linear":96849.0,"score_lmetric":108166}],"chosen_score_linear":-87952.0,"chosen_score_lmetric":6902,"t_first_token":349025.850135259,"t_first_token_unix":1779779890.864079,"t_done":349028.073861125,"t_done_unix":1779779893.0878046},{"request_id":"1364090:9:1423736:840","session_id":"1364090","input_length":101432,"t_proxy_recv":349027.424734845,"t_decision_unix":1779779892.4386759,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":97280,"affinity_cache_ratio":0.9590661724110734,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":97280,"estimated_new_tokens":4152,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":282847,"ongoing_decode_tokens":155359,"pending_prefill_tokens":5120,"num_requests":3,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":7680,"new_prefill":93752,"score_linear":275167.0,"score_lmetric":296616},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":101432,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2249,"cache_hit":0,"new_prefill":101432,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2153,"cache_hit":0,"new_prefill":101432,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":169296,"ongoing_decode_tokens":106143,"pending_prefill_tokens":14001,"num_requests":1,"active_p_offloads":0,"cached_blocks":2655,"cache_hit":7680,"new_prefill":93752,"score_linear":161616.0,"score_lmetric":107753},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":108166,"ongoing_decode_tokens":108166,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1733,"cache_hit":0,"new_prefill":101432,"score_linear":108166.0,"score_lmetric":101432},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":158384,"ongoing_decode_tokens":0,"pending_prefill_tokens":69296,"num_requests":2,"active_p_offloads":0,"cached_blocks":1562,"cache_hit":7680,"new_prefill":93752,"score_linear":150704.0,"score_lmetric":326096},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1667,"cache_hit":97280,"new_prefill":4152,"score_linear":-97280.0,"score_lmetric":0}],"chosen_score_linear":-97280.0,"chosen_score_lmetric":0,"t_first_token":349030.110988768,"t_first_token_unix":1779779895.1249318,"t_done":349033.727418613,"t_done_unix":1779779898.741362},{"request_id":"1371622:3:1443876:871","session_id":"1371622","input_length":19292,"t_proxy_recv":349028.348049058,"t_decision_unix":1779779893.3619905,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":18944,"affinity_cache_ratio":0.9819614347916235,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":18944,"estimated_new_tokens":348,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":282847,"ongoing_decode_tokens":155359,"pending_prefill_tokens":5120,"num_requests":3,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":0,"new_prefill":19292,"score_linear":282847.0,"score_lmetric":73236},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":19292,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2249,"cache_hit":0,"new_prefill":19292,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2153,"cache_hit":0,"new_prefill":19292,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":169296,"ongoing_decode_tokens":106143,"pending_prefill_tokens":14001,"num_requests":1,"active_p_offloads":0,"cached_blocks":2655,"cache_hit":11264,"new_prefill":8028,"score_linear":158032.0,"score_lmetric":22029},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1735,"cache_hit":18944,"new_prefill":348,"score_linear":-18944.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":158384,"ongoing_decode_tokens":0,"pending_prefill_tokens":69296,"num_requests":2,"active_p_offloads":0,"cached_blocks":1562,"cache_hit":0,"new_prefill":19292,"score_linear":158384.0,"score_lmetric":177176},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":101432,"ongoing_decode_tokens":0,"pending_prefill_tokens":4152,"num_requests":1,"active_p_offloads":0,"cached_blocks":1667,"cache_hit":0,"new_prefill":19292,"score_linear":101432.0,"score_lmetric":23444}],"chosen_score_linear":-18944.0,"chosen_score_lmetric":0,"t_first_token":349028.43497888,"t_first_token_unix":1779779893.4489224,"t_done":349034.930474514,"t_done_unix":1779779899.9444177},{"request_id":"1314357:12:1446681:873","session_id":"1314357","input_length":108409,"t_proxy_recv":349037.763056263,"t_decision_unix":1779779902.7769976,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":108032,"affinity_cache_ratio":0.996522428949626,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":108032,"estimated_new_tokens":377,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":282847,"ongoing_decode_tokens":155359,"pending_prefill_tokens":5120,"num_requests":3,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":0,"new_prefill":108409,"score_linear":282847.0,"score_lmetric":340587},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":108409,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2249,"cache_hit":0,"new_prefill":108409,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2153,"cache_hit":0,"new_prefill":108409,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":169296,"ongoing_decode_tokens":169296,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2655,"cache_hit":56832,"new_prefill":51577,"score_linear":112464.0,"score_lmetric":51577},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1737,"cache_hit":108032,"new_prefill":377,"score_linear":-108032.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":158384,"ongoing_decode_tokens":0,"pending_prefill_tokens":69296,"num_requests":2,"active_p_offloads":0,"cached_blocks":1562,"cache_hit":0,"new_prefill":108409,"score_linear":158384.0,"score_lmetric":355410},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":108020,"ongoing_decode_tokens":0,"pending_prefill_tokens":6644,"num_requests":1,"active_p_offloads":0,"cached_blocks":1675,"cache_hit":0,"new_prefill":108409,"score_linear":108020.0,"score_lmetric":115053}],"chosen_score_linear":-108032.0,"chosen_score_lmetric":0,"t_first_token":349037.972283727,"t_first_token_unix":1779779902.9862282,"t_done":349038.573086952,"t_done_unix":1779779903.587031},{"request_id":"1294611:12:1441557:868","session_id":"1294611","input_length":63153,"t_proxy_recv":349020.516446442,"t_decision_unix":1779779885.5303872,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":49152,"affinity_cache_ratio":0.7783003182746663,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":49152,"estimated_new_tokens":14001,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":282847,"ongoing_decode_tokens":155359,"pending_prefill_tokens":5120,"num_requests":3,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":0,"new_prefill":63153,"score_linear":282847.0,"score_lmetric":204819},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":63153,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2249,"cache_hit":0,"new_prefill":63153,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2153,"cache_hit":0,"new_prefill":63153,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2655,"cache_hit":49152,"new_prefill":14001,"score_linear":56991.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1722,"cache_hit":12800,"new_prefill":50353,"score_linear":-12800.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":158384,"ongoing_decode_tokens":0,"pending_prefill_tokens":69296,"num_requests":2,"active_p_offloads":0,"cached_blocks":1562,"cache_hit":0,"new_prefill":63153,"score_linear":158384.0,"score_lmetric":264898},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":96849,"ongoing_decode_tokens":0,"pending_prefill_tokens":10321,"num_requests":1,"active_p_offloads":0,"cached_blocks":1646,"cache_hit":12800,"new_prefill":50353,"score_linear":84049.0,"score_lmetric":60674}],"chosen_score_linear":56991.0,"chosen_score_lmetric":0,"t_first_token":349034.745570851,"t_first_token_unix":1779779899.7595143,"t_done":349039.105550195,"t_done_unix":1779779904.1194935},{"request_id":"1294611:13:1446349:872","session_id":"1294611","input_length":63488,"t_proxy_recv":349039.118027924,"t_decision_unix":1779779904.131969,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":62976,"affinity_cache_ratio":0.9919354838709677,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":62976,"estimated_new_tokens":512,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":282847,"ongoing_decode_tokens":155359,"pending_prefill_tokens":5120,"num_requests":3,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":0,"new_prefill":63488,"score_linear":282847.0,"score_lmetric":205824},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":63488,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2249,"cache_hit":0,"new_prefill":63488,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2153,"cache_hit":0,"new_prefill":63488,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2682,"cache_hit":62976,"new_prefill":512,"score_linear":43167.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1737,"cache_hit":12800,"new_prefill":50688,"score_linear":-12800.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":158384,"ongoing_decode_tokens":0,"pending_prefill_tokens":69296,"num_requests":2,"active_p_offloads":0,"cached_blocks":1562,"cache_hit":0,"new_prefill":63488,"score_linear":158384.0,"score_lmetric":265568},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":108020,"ongoing_decode_tokens":108020,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1675,"cache_hit":12800,"new_prefill":50688,"score_linear":95220.0,"score_lmetric":50688}],"chosen_score_linear":43167.0,"chosen_score_lmetric":0,"t_first_token":349039.23262404,"t_first_token_unix":1779779904.2465677,"t_done":349041.011911212,"t_done_unix":1779779906.0258548},{"request_id":"1370129:6:1394177:722","session_id":"1370129","input_length":106143,"t_proxy_recv":348976.940947696,"t_decision_unix":1779779841.9548893,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":88064,"affinity_cache_ratio":0.8296731767521174,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":88064,"estimated_new_tokens":18079,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":7680,"new_prefill":98463,"score_linear":-7680.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1881,"cache_hit":0,"new_prefill":106143,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":2245,"cache_hit":0,"new_prefill":106143,"score_linear":0.0,"score_lmetric":-106143},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2144,"cache_hit":0,"new_prefill":106143,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":116951,"ongoing_decode_tokens":116951,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2639,"cache_hit":88064,"new_prefill":18079,"score_linear":28887.0,"score_lmetric":18079},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":220659,"ongoing_decode_tokens":220659,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1712,"cache_hit":0,"new_prefill":106143,"score_linear":220659.0,"score_lmetric":318429},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1562,"cache_hit":7680,"new_prefill":98463,"score_linear":-7680.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":317459,"ongoing_decode_tokens":124090,"pending_prefill_tokens":14169,"num_requests":3,"active_p_offloads":0,"cached_blocks":1376,"cache_hit":7680,"new_prefill":98463,"score_linear":309779.0,"score_lmetric":337896}],"chosen_score_linear":28887.0,"chosen_score_lmetric":18079,"t_first_token":348987.919296001,"t_first_token_unix":1779779852.9332395,"t_done":349044.223676046,"t_done_unix":1779779909.2376194},{"request_id":"1313181:9:1333349:415","session_id":"1313181","input_length":129673,"t_proxy_recv":349002.483868234,"t_decision_unix":1779779867.4978094,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":7,"chosen_idx":6,"affinity_cache_hit":62464,"affinity_cache_ratio":0.481703978468918,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":61952,"estimated_new_tokens":67721,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":155359,"ongoing_decode_tokens":125947,"pending_prefill_tokens":6884,"num_requests":2,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":24064,"new_prefill":105609,"score_linear":131295.0,"score_lmetric":224986},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1881,"cache_hit":0,"new_prefill":129673,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2249,"cache_hit":0,"new_prefill":129673,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":75718,"ongoing_decode_tokens":1840,"pending_prefill_tokens":2710,"num_requests":2,"active_p_offloads":0,"cached_blocks":2144,"cache_hit":0,"new_prefill":129673,"score_linear":75718.0,"score_lmetric":264766},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2655,"cache_hit":0,"new_prefill":129673,"score_linear":106143.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1722,"cache_hit":0,"new_prefill":129673,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1562,"cache_hit":61952,"new_prefill":67721,"score_linear":-61952.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":73392,"ongoing_decode_tokens":73392,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1600,"cache_hit":62464,"new_prefill":67209,"score_linear":10928.0,"score_lmetric":67209}],"chosen_score_linear":-61952.0,"chosen_score_lmetric":0,"t_first_token":349042.597549251,"t_first_token_unix":1779779907.611493,"t_done":349044.618053892,"t_done_unix":1779779909.6319973},{"request_id":"1342921:6:1448548:876","session_id":"1342921","input_length":75761,"t_proxy_recv":349043.979839618,"t_decision_unix":1779779908.9937809,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":73728,"affinity_cache_ratio":0.9731656129142963,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":73728,"estimated_new_tokens":2033,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":282847,"ongoing_decode_tokens":155359,"pending_prefill_tokens":5120,"num_requests":3,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":0,"new_prefill":75761,"score_linear":282847.0,"score_lmetric":242643},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":75761,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2249,"cache_hit":0,"new_prefill":75761,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2153,"cache_hit":73728,"new_prefill":2033,"score_linear":-73728.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2683,"cache_hit":0,"new_prefill":75761,"score_linear":106143.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":108791,"ongoing_decode_tokens":0,"pending_prefill_tokens":759,"num_requests":1,"active_p_offloads":0,"cached_blocks":1737,"cache_hit":0,"new_prefill":75761,"score_linear":108791.0,"score_lmetric":76520},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":158384,"ongoing_decode_tokens":158384,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1562,"cache_hit":0,"new_prefill":75761,"score_linear":158384.0,"score_lmetric":151522},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":108020,"ongoing_decode_tokens":108020,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1675,"cache_hit":0,"new_prefill":75761,"score_linear":108020.0,"score_lmetric":75761}],"chosen_score_linear":-73728.0,"chosen_score_lmetric":0,"t_first_token":349044.944448308,"t_first_token_unix":1779779909.958392,"t_done":349045.313580322,"t_done_unix":1779779910.327524},{"request_id":"1314357:13:1448346:875","session_id":"1314357","input_length":108791,"t_proxy_recv":349043.452653857,"t_decision_unix":1779779908.4665952,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":108032,"affinity_cache_ratio":0.9930233199437454,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":108032,"estimated_new_tokens":759,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":282847,"ongoing_decode_tokens":155359,"pending_prefill_tokens":5120,"num_requests":3,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":0,"new_prefill":108791,"score_linear":282847.0,"score_lmetric":341733},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":108791,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2249,"cache_hit":0,"new_prefill":108791,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2153,"cache_hit":0,"new_prefill":108791,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2683,"cache_hit":56832,"new_prefill":51959,"score_linear":49311.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1737,"cache_hit":108032,"new_prefill":759,"score_linear":-108032.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":158384,"ongoing_decode_tokens":158384,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1562,"cache_hit":0,"new_prefill":108791,"score_linear":158384.0,"score_lmetric":217582},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":108020,"ongoing_decode_tokens":108020,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1675,"cache_hit":0,"new_prefill":108791,"score_linear":108020.0,"score_lmetric":108791}],"chosen_score_linear":-108032.0,"chosen_score_lmetric":0,"t_first_token":349043.986727998,"t_first_token_unix":1779779909.0006711,"t_done":349045.778754448,"t_done_unix":1779779910.7926977},{"request_id":"1363943:13:1438562:865","session_id":"1363943","input_length":28711,"t_proxy_recv":349010.83586655,"t_decision_unix":1779779875.8498075,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":27136,"affinity_cache_ratio":0.9451429765595069,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":27136,"estimated_new_tokens":1575,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":282847,"ongoing_decode_tokens":155359,"pending_prefill_tokens":5120,"num_requests":3,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":8704,"new_prefill":20007,"score_linear":274143.0,"score_lmetric":75381},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1881,"cache_hit":0,"new_prefill":28711,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2249,"cache_hit":0,"new_prefill":28711,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":75718,"ongoing_decode_tokens":1840,"pending_prefill_tokens":2710,"num_requests":2,"active_p_offloads":0,"cached_blocks":2144,"cache_hit":0,"new_prefill":28711,"score_linear":75718.0,"score_lmetric":62842},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2655,"cache_hit":4096,"new_prefill":24615,"score_linear":102047.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1722,"cache_hit":0,"new_prefill":28711,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":129673,"ongoing_decode_tokens":0,"pending_prefill_tokens":67721,"num_requests":1,"active_p_offloads":0,"cached_blocks":1562,"cache_hit":27136,"new_prefill":1575,"score_linear":102537.0,"score_lmetric":69296},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":86193,"ongoing_decode_tokens":0,"pending_prefill_tokens":12465,"num_requests":1,"active_p_offloads":0,"cached_blocks":1621,"cache_hit":4096,"new_prefill":24615,"score_linear":82097.0,"score_lmetric":37080}],"chosen_score_linear":102537.0,"chosen_score_lmetric":69296,"t_first_token":349042.599104965,"t_first_token_unix":1779779907.613048,"t_done":349048.461067291,"t_done_unix":1779779913.4750106},{"request_id":"1294611:14:1449074:877","session_id":"1294611","input_length":63616,"t_proxy_recv":349045.804965345,"t_decision_unix":1779779910.8189068,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":63488,"affinity_cache_ratio":0.9979879275653923,"affinity_num_requests":-1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":63488,"estimated_new_tokens":128,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":282847,"ongoing_decode_tokens":155359,"pending_prefill_tokens":5120,"num_requests":3,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":0,"new_prefill":63616,"score_linear":282847.0,"score_lmetric":206208},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":63616,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2249,"cache_hit":0,"new_prefill":63616,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2157,"cache_hit":0,"new_prefill":63616,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":2719,"cache_hit":63488,"new_prefill":128,"score_linear":-63488.0,"score_lmetric":-128},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1738,"cache_hit":12800,"new_prefill":50816,"score_linear":-12800.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":91224,"ongoing_decode_tokens":91224,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1580,"cache_hit":0,"new_prefill":63616,"score_linear":91224.0,"score_lmetric":127232},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":108020,"ongoing_decode_tokens":108020,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1675,"cache_hit":12800,"new_prefill":50816,"score_linear":95220.0,"score_lmetric":50816}],"chosen_score_linear":-63488.0,"chosen_score_lmetric":-128,"t_first_token":349045.912677288,"t_first_token_unix":1779779910.9266205,"t_done":349050.771048054,"t_done_unix":1779779915.784992},{"request_id":"1314357:14:1450740:878","session_id":"1314357","input_length":109010,"t_proxy_recv":349051.255984025,"t_decision_unix":1779779916.2699254,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":108544,"affinity_cache_ratio":0.9957251628290983,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":108544,"estimated_new_tokens":466,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":282847,"ongoing_decode_tokens":155359,"pending_prefill_tokens":5120,"num_requests":3,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":0,"new_prefill":109010,"score_linear":282847.0,"score_lmetric":342390},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":109010,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2249,"cache_hit":0,"new_prefill":109010,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2157,"cache_hit":0,"new_prefill":109010,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":2720,"cache_hit":56832,"new_prefill":52178,"score_linear":-56832.0,"score_lmetric":-52178},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1738,"cache_hit":108544,"new_prefill":466,"score_linear":-108544.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":62513,"ongoing_decode_tokens":62513,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1584,"cache_hit":0,"new_prefill":109010,"score_linear":62513.0,"score_lmetric":109010},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":108020,"ongoing_decode_tokens":108020,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1675,"cache_hit":0,"new_prefill":109010,"score_linear":108020.0,"score_lmetric":109010}],"chosen_score_linear":-108544.0,"chosen_score_lmetric":0,"t_first_token":349051.483543431,"t_first_token_unix":1779779916.4974868,"t_done":349052.766689549,"t_done_unix":1779779917.780633},{"request_id":"1371622:4:1452210:881","session_id":"1371622","input_length":20439,"t_proxy_recv":349056.226738363,"t_decision_unix":1779779921.24068,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":19968,"affinity_cache_ratio":0.9769558197563482,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":19968,"estimated_new_tokens":471,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":282847,"ongoing_decode_tokens":155359,"pending_prefill_tokens":5120,"num_requests":3,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":0,"new_prefill":20439,"score_linear":282847.0,"score_lmetric":76677},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55050,"ongoing_decode_tokens":0,"pending_prefill_tokens":2314,"num_requests":1,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":20439,"score_linear":55050.0,"score_lmetric":22753},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2249,"cache_hit":0,"new_prefill":20439,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2157,"cache_hit":0,"new_prefill":20439,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":2720,"cache_hit":11264,"new_prefill":9175,"score_linear":-11264.0,"score_lmetric":-9175},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1739,"cache_hit":19968,"new_prefill":471,"score_linear":-19968.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":62513,"ongoing_decode_tokens":62513,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1584,"cache_hit":0,"new_prefill":20439,"score_linear":62513.0,"score_lmetric":20439},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":108020,"ongoing_decode_tokens":108020,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1675,"cache_hit":0,"new_prefill":20439,"score_linear":108020.0,"score_lmetric":20439}],"chosen_score_linear":-19968.0,"chosen_score_lmetric":0,"t_first_token":349056.286291218,"t_first_token_unix":1779779921.300235,"t_done":349057.206330487,"t_done_unix":1779779922.2202737},{"request_id":"1268861:11:1452208:880","session_id":"1268861","input_length":55050,"t_proxy_recv":349056.219279324,"t_decision_unix":1779779921.2332206,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":52736,"affinity_cache_ratio":0.9579654859218892,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":52736,"estimated_new_tokens":2314,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":282847,"ongoing_decode_tokens":155359,"pending_prefill_tokens":5120,"num_requests":3,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":0,"new_prefill":55050,"score_linear":282847.0,"score_lmetric":180510},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":52736,"new_prefill":2314,"score_linear":-52736.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2249,"cache_hit":0,"new_prefill":55050,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2157,"cache_hit":0,"new_prefill":55050,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":2720,"cache_hit":0,"new_prefill":55050,"score_linear":0.0,"score_lmetric":-55050},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1739,"cache_hit":0,"new_prefill":55050,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":62513,"ongoing_decode_tokens":62513,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1584,"cache_hit":0,"new_prefill":55050,"score_linear":62513.0,"score_lmetric":55050},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":108020,"ongoing_decode_tokens":108020,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1675,"cache_hit":12288,"new_prefill":42762,"score_linear":95732.0,"score_lmetric":42762}],"chosen_score_linear":-52736.0,"chosen_score_lmetric":0,"t_first_token":349057.048369256,"t_first_token_unix":1779779922.0623133,"t_done":349057.53124666,"t_done_unix":1779779922.54519},{"request_id":"1314357:15:1452292:882","session_id":"1314357","input_length":109164,"t_proxy_recv":349056.480890498,"t_decision_unix":1779779921.4948318,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":109056,"affinity_cache_ratio":0.9990106628558866,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":109056,"estimated_new_tokens":108,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":282847,"ongoing_decode_tokens":155359,"pending_prefill_tokens":5120,"num_requests":3,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":0,"new_prefill":109164,"score_linear":282847.0,"score_lmetric":342852},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":55050,"ongoing_decode_tokens":0,"pending_prefill_tokens":2314,"num_requests":1,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":109164,"score_linear":55050.0,"score_lmetric":111478},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2249,"cache_hit":0,"new_prefill":109164,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2157,"cache_hit":0,"new_prefill":109164,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":2720,"cache_hit":56832,"new_prefill":52332,"score_linear":-56832.0,"score_lmetric":-52332},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":20439,"ongoing_decode_tokens":20439,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1739,"cache_hit":109056,"new_prefill":108,"score_linear":-88617.0,"score_lmetric":108},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":62513,"ongoing_decode_tokens":62513,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1584,"cache_hit":0,"new_prefill":109164,"score_linear":62513.0,"score_lmetric":109164},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":108020,"ongoing_decode_tokens":108020,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1675,"cache_hit":0,"new_prefill":109164,"score_linear":108020.0,"score_lmetric":109164}],"chosen_score_linear":-88617.0,"chosen_score_lmetric":108,"t_first_token":349056.6602646,"t_first_token_unix":1779779921.6742089,"t_done":349060.725318182,"t_done_unix":1779779925.7392616},{"request_id":"1313181:9:1386945:689","session_id":"1313181","input_length":62513,"t_proxy_recv":349044.62872713,"t_decision_unix":1779779909.6426687,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":62464,"affinity_cache_ratio":0.999216163038088,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":62464,"estimated_new_tokens":49,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":282847,"ongoing_decode_tokens":155359,"pending_prefill_tokens":5120,"num_requests":3,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":24064,"new_prefill":38449,"score_linear":258783.0,"score_lmetric":130707},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":62513,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2249,"cache_hit":0,"new_prefill":62513,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":75761,"ongoing_decode_tokens":0,"pending_prefill_tokens":2033,"num_requests":1,"active_p_offloads":0,"cached_blocks":2153,"cache_hit":0,"new_prefill":62513,"score_linear":75761.0,"score_lmetric":64546},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":2719,"cache_hit":0,"new_prefill":62513,"score_linear":0.0,"score_lmetric":-62513},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":108791,"ongoing_decode_tokens":108791,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1737,"cache_hit":0,"new_prefill":62513,"score_linear":108791.0,"score_lmetric":62513},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":28711,"ongoing_decode_tokens":28711,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1580,"cache_hit":62464,"new_prefill":49,"score_linear":-33753.0,"score_lmetric":49},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":108020,"ongoing_decode_tokens":108020,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1675,"cache_hit":62464,"new_prefill":49,"score_linear":45556.0,"score_lmetric":49}],"chosen_score_linear":-33753.0,"chosen_score_lmetric":49,"t_first_token":349044.71540426,"t_first_token_unix":1779779909.7293472,"t_done":349062.726588591,"t_done_unix":1779779927.7405322},{"request_id":"1371622:5:1455608:884","session_id":"1371622","input_length":21518,"t_proxy_recv":349068.006687056,"t_decision_unix":1779779933.020628,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":20480,"affinity_cache_ratio":0.951761316107445,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":20480,"estimated_new_tokens":1038,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":282847,"ongoing_decode_tokens":155359,"pending_prefill_tokens":5120,"num_requests":3,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":0,"new_prefill":21518,"score_linear":282847.0,"score_lmetric":79914},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1892,"cache_hit":0,"new_prefill":21518,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2249,"cache_hit":0,"new_prefill":21518,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2157,"cache_hit":0,"new_prefill":21518,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2720,"cache_hit":11264,"new_prefill":10254,"score_linear":-11264.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1740,"cache_hit":20480,"new_prefill":1038,"score_linear":-20480.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":129839,"ongoing_decode_tokens":0,"pending_prefill_tokens":65327,"num_requests":1,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":21518,"score_linear":129839.0,"score_lmetric":86845},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":108020,"ongoing_decode_tokens":108020,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1675,"cache_hit":0,"new_prefill":21518,"score_linear":108020.0,"score_lmetric":21518}],"chosen_score_linear":-20480.0,"chosen_score_lmetric":0,"t_first_token":349068.2475745,"t_first_token_unix":1779779933.261518,"t_done":349068.699251082,"t_done_unix":1779779933.7131941},{"request_id":"1264679:3:1431512:853","session_id":"1264679","input_length":125947,"t_proxy_recv":348986.256443264,"t_decision_unix":1779779851.2703846,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":114176,"affinity_cache_ratio":0.9065400525617918,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":114176,"estimated_new_tokens":11771,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":114176,"new_prefill":11771,"score_linear":-114176.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1881,"cache_hit":0,"new_prefill":125947,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":-1,"active_p_offloads":0,"cached_blocks":2245,"cache_hit":0,"new_prefill":125947,"score_linear":0.0,"score_lmetric":-125947},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":1840,"ongoing_decode_tokens":1840,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2144,"cache_hit":0,"new_prefill":125947,"score_linear":1840.0,"score_lmetric":125947},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":223094,"ongoing_decode_tokens":116951,"pending_prefill_tokens":18079,"num_requests":2,"active_p_offloads":0,"cached_blocks":2639,"cache_hit":0,"new_prefill":125947,"score_linear":223094.0,"score_lmetric":288052},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":178887,"ongoing_decode_tokens":178887,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1717,"cache_hit":0,"new_prefill":125947,"score_linear":178887.0,"score_lmetric":251894},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1562,"cache_hit":0,"new_prefill":125947,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":317459,"ongoing_decode_tokens":124090,"pending_prefill_tokens":14169,"num_requests":3,"active_p_offloads":0,"cached_blocks":1376,"cache_hit":0,"new_prefill":125947,"score_linear":317459.0,"score_lmetric":420348}],"chosen_score_linear":-114176.0,"chosen_score_lmetric":0,"t_first_token":348994.745874228,"t_first_token_unix":1779779859.7598183,"t_done":349079.507172145,"t_done_unix":1779779944.5211155},{"request_id":"1364090:10:1426859:847","session_id":"1364090","input_length":108020,"t_proxy_recv":349033.747020315,"t_decision_unix":1779779898.7609613,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":101376,"affinity_cache_ratio":0.9384928716904277,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":101376,"estimated_new_tokens":6644,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":282847,"ongoing_decode_tokens":155359,"pending_prefill_tokens":5120,"num_requests":3,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":7680,"new_prefill":100340,"score_linear":275167.0,"score_lmetric":316380},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1888,"cache_hit":0,"new_prefill":108020,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2249,"cache_hit":0,"new_prefill":108020,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2153,"cache_hit":0,"new_prefill":108020,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":169296,"ongoing_decode_tokens":106143,"pending_prefill_tokens":14001,"num_requests":1,"active_p_offloads":0,"cached_blocks":2655,"cache_hit":7680,"new_prefill":100340,"score_linear":161616.0,"score_lmetric":114341},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":19292,"ongoing_decode_tokens":19292,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1735,"cache_hit":0,"new_prefill":108020,"score_linear":19292.0,"score_lmetric":108020},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":158384,"ongoing_decode_tokens":0,"pending_prefill_tokens":69296,"num_requests":2,"active_p_offloads":0,"cached_blocks":1562,"cache_hit":7680,"new_prefill":100340,"score_linear":150704.0,"score_lmetric":339272},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1675,"cache_hit":101376,"new_prefill":6644,"score_linear":-101376.0,"score_lmetric":0}],"chosen_score_linear":-101376.0,"chosen_score_lmetric":0,"t_first_token":349037.914735782,"t_first_token_unix":1779779902.92868,"t_done":349088.425695443,"t_done_unix":1779779953.4396439},{"request_id":"1356885:2:1438037:864","session_id":"1356885","input_length":127488,"t_proxy_recv":349009.132735597,"t_decision_unix":1779779874.1466765,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":122368,"affinity_cache_ratio":0.9598393574297188,"affinity_num_requests":2,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":122368,"estimated_new_tokens":5120,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":155359,"ongoing_decode_tokens":155359,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":122368,"new_prefill":5120,"score_linear":32991.0,"score_lmetric":10240},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1881,"cache_hit":1024,"new_prefill":126464,"score_linear":-1024.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2249,"cache_hit":0,"new_prefill":127488,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":75718,"ongoing_decode_tokens":1840,"pending_prefill_tokens":2710,"num_requests":2,"active_p_offloads":0,"cached_blocks":2144,"cache_hit":0,"new_prefill":127488,"score_linear":75718.0,"score_lmetric":260396},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2655,"cache_hit":6656,"new_prefill":120832,"score_linear":99487.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1722,"cache_hit":0,"new_prefill":127488,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":129673,"ongoing_decode_tokens":0,"pending_prefill_tokens":67721,"num_requests":1,"active_p_offloads":0,"cached_blocks":1562,"cache_hit":0,"new_prefill":127488,"score_linear":129673.0,"score_lmetric":195209},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":86193,"ongoing_decode_tokens":0,"pending_prefill_tokens":12465,"num_requests":1,"active_p_offloads":0,"cached_blocks":1621,"cache_hit":0,"new_prefill":127488,"score_linear":86193.0,"score_lmetric":139953}],"chosen_score_linear":32991.0,"chosen_score_lmetric":10240,"t_first_token":349084.142279368,"t_first_token_unix":1779779949.156224,"t_done":349089.875429415,"t_done_unix":1779779954.8893728},{"request_id":"1242838:5:1460026:886","session_id":"1242838","input_length":26406,"t_proxy_recv":349082.442457593,"t_decision_unix":1779779947.4563992,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":26112,"affinity_cache_ratio":0.9888661667802772,"affinity_num_requests":2,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":26112,"estimated_new_tokens":294,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":156900,"ongoing_decode_tokens":29412,"pending_prefill_tokens":5120,"num_requests":2,"active_p_offloads":0,"cached_blocks":1650,"cache_hit":26112,"new_prefill":294,"score_linear":130788.0,"score_lmetric":10828},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1892,"cache_hit":0,"new_prefill":26406,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2249,"cache_hit":0,"new_prefill":26406,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2157,"cache_hit":0,"new_prefill":26406,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2720,"cache_hit":0,"new_prefill":26406,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1742,"cache_hit":0,"new_prefill":26406,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":159107,"ongoing_decode_tokens":0,"pending_prefill_tokens":65411,"num_requests":2,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":26406,"score_linear":159107.0,"score_lmetric":183634},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":108020,"ongoing_decode_tokens":108020,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1675,"cache_hit":0,"new_prefill":26406,"score_linear":108020.0,"score_lmetric":26406}],"chosen_score_linear":130788.0,"chosen_score_lmetric":10828,"t_first_token":349087.349640818,"t_first_token_unix":1779779952.363584,"t_done":349093.66315997,"t_done_unix":1779779958.6771033},{"request_id":"1363093:6:1460799:888","session_id":"1363093","input_length":81299,"t_proxy_recv":349085.023819715,"t_decision_unix":1779779950.037761,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":73216,"affinity_cache_ratio":0.9005768828644879,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":73216,"estimated_new_tokens":8083,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":183306,"ongoing_decode_tokens":156900,"pending_prefill_tokens":294,"num_requests":3,"active_p_offloads":0,"cached_blocks":1650,"cache_hit":0,"new_prefill":81299,"score_linear":183306.0,"score_lmetric":244779},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1892,"cache_hit":0,"new_prefill":81299,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2249,"cache_hit":0,"new_prefill":81299,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2157,"cache_hit":0,"new_prefill":81299,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2720,"cache_hit":73216,"new_prefill":8083,"score_linear":-73216.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1742,"cache_hit":0,"new_prefill":81299,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":159107,"ongoing_decode_tokens":0,"pending_prefill_tokens":65411,"num_requests":2,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":81299,"score_linear":159107.0,"score_lmetric":293420},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":108020,"ongoing_decode_tokens":108020,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1675,"cache_hit":0,"new_prefill":81299,"score_linear":108020.0,"score_lmetric":81299}],"chosen_score_linear":-73216.0,"chosen_score_lmetric":0,"t_first_token":349093.886108354,"t_first_token_unix":1779779958.9000516,"t_done":349094.514633102,"t_done_unix":1779779959.528578},{"request_id":"1268861:12:1463977:891","session_id":"1268861","input_length":56147,"t_proxy_recv":349095.632052821,"t_decision_unix":1779779960.645994,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":54784,"affinity_cache_ratio":0.9757244376369174,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":54784,"estimated_new_tokens":1363,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":157975,"ongoing_decode_tokens":157975,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1660,"cache_hit":0,"new_prefill":56147,"score_linear":157975.0,"score_lmetric":112294},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1892,"cache_hit":54784,"new_prefill":1363,"score_linear":-54784.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2249,"cache_hit":0,"new_prefill":56147,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2157,"cache_hit":0,"new_prefill":56147,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":0,"new_prefill":56147,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1742,"cache_hit":0,"new_prefill":56147,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":159107,"ongoing_decode_tokens":0,"pending_prefill_tokens":65411,"num_requests":2,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":56147,"score_linear":159107.0,"score_lmetric":243116},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1688,"cache_hit":12288,"new_prefill":43859,"score_linear":-12288.0,"score_lmetric":0}],"chosen_score_linear":-54784.0,"chosen_score_lmetric":0,"t_first_token":349096.152410668,"t_first_token_unix":1779779961.166354,"t_done":349097.059852047,"t_done_unix":1779779962.0737958},{"request_id":"1404395:2:1435941:862","session_id":"1404395","input_length":29412,"t_proxy_recv":349002.025134123,"t_decision_unix":1779779867.0390756,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":22528,"affinity_cache_ratio":0.7659458724330205,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":22528,"estimated_new_tokens":6884,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":125947,"ongoing_decode_tokens":125947,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":22528,"new_prefill":6884,"score_linear":103419.0,"score_lmetric":6884},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1881,"cache_hit":0,"new_prefill":29412,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2249,"cache_hit":0,"new_prefill":29412,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":75718,"ongoing_decode_tokens":1840,"pending_prefill_tokens":2710,"num_requests":2,"active_p_offloads":0,"cached_blocks":2144,"cache_hit":0,"new_prefill":29412,"score_linear":75718.0,"score_lmetric":64244},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":106143,"ongoing_decode_tokens":106143,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2655,"cache_hit":0,"new_prefill":29412,"score_linear":106143.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":106959,"ongoing_decode_tokens":106959,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1721,"cache_hit":512,"new_prefill":28900,"score_linear":106447.0,"score_lmetric":28900},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1562,"cache_hit":0,"new_prefill":29412,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":255813,"ongoing_decode_tokens":255813,"pending_prefill_tokens":0,"num_requests":3,"active_p_offloads":0,"cached_blocks":1592,"cache_hit":0,"new_prefill":29412,"score_linear":255813.0,"score_lmetric":88236}],"chosen_score_linear":103419.0,"chosen_score_lmetric":6884,"t_first_token":349003.357124299,"t_first_token_unix":1779779868.3710682,"t_done":349099.491984678,"t_done_unix":1779779964.505928},{"request_id":"1371622:6:1465333:892","session_id":"1371622","input_length":23079,"t_proxy_recv":349099.88752521,"t_decision_unix":1779779964.9014668,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":21504,"affinity_cache_ratio":0.9317561419472248,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":21504,"estimated_new_tokens":1575,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":162955,"ongoing_decode_tokens":128563,"pending_prefill_tokens":2648,"num_requests":2,"active_p_offloads":0,"cached_blocks":1675,"cache_hit":0,"new_prefill":23079,"score_linear":162955.0,"score_lmetric":51454},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1894,"cache_hit":0,"new_prefill":23079,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2249,"cache_hit":0,"new_prefill":23079,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2157,"cache_hit":0,"new_prefill":23079,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":11264,"new_prefill":11815,"score_linear":-11264.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1742,"cache_hit":21504,"new_prefill":1575,"score_linear":-21504.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":159107,"ongoing_decode_tokens":0,"pending_prefill_tokens":65411,"num_requests":2,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":23079,"score_linear":159107.0,"score_lmetric":176980},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1688,"cache_hit":0,"new_prefill":23079,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-21504.0,"chosen_score_lmetric":0,"t_first_token":349100.199943478,"t_first_token_unix":1779779965.2138867,"t_done":349100.723133841,"t_done_unix":1779779965.737077},{"request_id":"1313181:10:1338837:436","session_id":"1313181","input_length":129839,"t_proxy_recv":349062.748750076,"t_decision_unix":1779779927.7626915,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":64512,"affinity_cache_ratio":0.4968614977009989,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":64512,"estimated_new_tokens":65327,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":282847,"ongoing_decode_tokens":155359,"pending_prefill_tokens":5120,"num_requests":3,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":24064,"new_prefill":105775,"score_linear":258783.0,"score_lmetric":332685},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1892,"cache_hit":0,"new_prefill":129839,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2249,"cache_hit":0,"new_prefill":129839,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2157,"cache_hit":0,"new_prefill":129839,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2720,"cache_hit":0,"new_prefill":129839,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1740,"cache_hit":0,"new_prefill":129839,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":64512,"new_prefill":65327,"score_linear":-64512.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":108020,"ongoing_decode_tokens":108020,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1675,"cache_hit":62464,"new_prefill":67375,"score_linear":45556.0,"score_lmetric":67375}],"chosen_score_linear":-64512.0,"chosen_score_lmetric":0,"t_first_token":349101.582007535,"t_first_token_unix":1779779966.595952,"t_done":349102.622220466,"t_done_unix":1779779967.636164},{"request_id":"1363943:14:1458616:885","session_id":"1363943","input_length":29268,"t_proxy_recv":349077.952900466,"t_decision_unix":1779779942.9668422,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":29184,"affinity_cache_ratio":0.997129971299713,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":29184,"estimated_new_tokens":84,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":282847,"ongoing_decode_tokens":155359,"pending_prefill_tokens":5120,"num_requests":3,"active_p_offloads":0,"cached_blocks":1626,"cache_hit":8704,"new_prefill":20564,"score_linear":274143.0,"score_lmetric":77052},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1892,"cache_hit":0,"new_prefill":29268,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2249,"cache_hit":0,"new_prefill":29268,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2157,"cache_hit":0,"new_prefill":29268,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2720,"cache_hit":4096,"new_prefill":25172,"score_linear":-4096.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1742,"cache_hit":0,"new_prefill":29268,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":129839,"ongoing_decode_tokens":0,"pending_prefill_tokens":65327,"num_requests":1,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":29184,"new_prefill":84,"score_linear":100655.0,"score_lmetric":65411},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":108020,"ongoing_decode_tokens":108020,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1675,"cache_hit":4096,"new_prefill":25172,"score_linear":103924.0,"score_lmetric":25172}],"chosen_score_linear":100655.0,"chosen_score_lmetric":65411,"t_first_token":349101.583570367,"t_first_token_unix":1779779966.5975134,"t_done":349102.726733986,"t_done_unix":1779779967.7406769},{"request_id":"1363943:15:1461716:889","session_id":"1363943","input_length":29651,"t_proxy_recv":349102.72972953,"t_decision_unix":1779779967.7436712,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":29184,"affinity_cache_ratio":0.984250109608445,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":29184,"estimated_new_tokens":467,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":162955,"ongoing_decode_tokens":162955,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1675,"cache_hit":8704,"new_prefill":20947,"score_linear":154251.0,"score_lmetric":41894},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1894,"cache_hit":0,"new_prefill":29651,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2249,"cache_hit":0,"new_prefill":29651,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2157,"cache_hit":0,"new_prefill":29651,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":4096,"new_prefill":25555,"score_linear":-4096.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1745,"cache_hit":0,"new_prefill":29651,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":64724,"ongoing_decode_tokens":64724,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1589,"cache_hit":29184,"new_prefill":467,"score_linear":35540.0,"score_lmetric":467},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1688,"cache_hit":4096,"new_prefill":25555,"score_linear":-4096.0,"score_lmetric":0}],"chosen_score_linear":35540.0,"chosen_score_lmetric":467,"t_first_token":349102.900378564,"t_first_token_unix":1779779967.914322,"t_done":349105.143308946,"t_done_unix":1779779970.157252},{"request_id":"1404395:3:1460072:887","session_id":"1404395","input_length":34392,"t_proxy_recv":349099.501317122,"t_decision_unix":1779779964.5152588,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":31744,"affinity_cache_ratio":0.9230053500814143,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":31744,"estimated_new_tokens":2648,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":128563,"ongoing_decode_tokens":128563,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1675,"cache_hit":31744,"new_prefill":2648,"score_linear":96819.0,"score_lmetric":2648},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1894,"cache_hit":0,"new_prefill":34392,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2249,"cache_hit":0,"new_prefill":34392,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2157,"cache_hit":0,"new_prefill":34392,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":0,"new_prefill":34392,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1742,"cache_hit":512,"new_prefill":33880,"score_linear":-512.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":159107,"ongoing_decode_tokens":0,"pending_prefill_tokens":65411,"num_requests":2,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":34392,"score_linear":159107.0,"score_lmetric":199606},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1688,"cache_hit":0,"new_prefill":34392,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":96819.0,"chosen_score_lmetric":2648,"t_first_token":349100.123478755,"t_first_token_unix":1779779965.137422,"t_done":349105.32729689,"t_done_unix":1779779970.3412402},{"request_id":"1313181:10:1399867:746","session_id":"1313181","input_length":64724,"t_proxy_recv":349102.632927697,"t_decision_unix":1779779967.6468692,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":64512,"affinity_cache_ratio":0.9967245534886595,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":64512,"estimated_new_tokens":212,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":162955,"ongoing_decode_tokens":162955,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1675,"cache_hit":24064,"new_prefill":40660,"score_linear":138891.0,"score_lmetric":81320},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1894,"cache_hit":0,"new_prefill":64724,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2249,"cache_hit":0,"new_prefill":64724,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2157,"cache_hit":0,"new_prefill":64724,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":0,"new_prefill":64724,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1745,"cache_hit":0,"new_prefill":64724,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":29268,"ongoing_decode_tokens":29268,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1589,"cache_hit":64512,"new_prefill":212,"score_linear":-35244.0,"score_lmetric":212},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1688,"cache_hit":62464,"new_prefill":2260,"score_linear":-62464.0,"score_lmetric":0}],"chosen_score_linear":-35244.0,"chosen_score_lmetric":212,"t_first_token":349102.720306724,"t_first_token_unix":1779779967.7342498,"t_done":349107.902905681,"t_done_unix":1779779972.9168491},{"request_id":"1404395:4:1467473:894","session_id":"1404395","input_length":34863,"t_proxy_recv":349107.34212244,"t_decision_unix":1779779972.3560638,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":34304,"affinity_cache_ratio":0.9839658090238935,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":34304,"estimated_new_tokens":559,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":128563,"ongoing_decode_tokens":128563,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1680,"cache_hit":34304,"new_prefill":559,"score_linear":94259.0,"score_lmetric":559},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1894,"cache_hit":0,"new_prefill":34863,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2249,"cache_hit":0,"new_prefill":34863,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2157,"cache_hit":0,"new_prefill":34863,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":0,"new_prefill":34863,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1745,"cache_hit":512,"new_prefill":34351,"score_linear":-512.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":99395,"ongoing_decode_tokens":99395,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1590,"cache_hit":0,"new_prefill":34863,"score_linear":99395.0,"score_lmetric":69726},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1688,"cache_hit":0,"new_prefill":34863,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":94259.0,"chosen_score_lmetric":559,"t_first_token":349107.468563546,"t_first_token_unix":1779779972.4825068,"t_done":349110.159068214,"t_done_unix":1779779975.1730118},{"request_id":"1371622:7:1468305:895","session_id":"1371622","input_length":24036,"t_proxy_recv":349110.111174054,"t_decision_unix":1779779975.1251154,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":23040,"affinity_cache_ratio":0.9585621567648527,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":23040,"estimated_new_tokens":996,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":163426,"ongoing_decode_tokens":163426,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1680,"cache_hit":0,"new_prefill":24036,"score_linear":163426.0,"score_lmetric":48072},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1894,"cache_hit":0,"new_prefill":24036,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2249,"cache_hit":0,"new_prefill":24036,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2157,"cache_hit":0,"new_prefill":24036,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":11264,"new_prefill":12772,"score_linear":-11264.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1745,"cache_hit":23040,"new_prefill":996,"score_linear":-23040.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":164602,"ongoing_decode_tokens":34671,"pending_prefill_tokens":64907,"num_requests":2,"active_p_offloads":0,"cached_blocks":1591,"cache_hit":0,"new_prefill":24036,"score_linear":164602.0,"score_lmetric":177886},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1688,"cache_hit":0,"new_prefill":24036,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-23040.0,"chosen_score_lmetric":0,"t_first_token":349110.330426042,"t_first_token_unix":1779779975.3443692,"t_done":349111.177683597,"t_done_unix":1779779976.1916268},{"request_id":"1356885:3:1451307:879","session_id":"1356885","input_length":128563,"t_proxy_recv":349089.896086578,"t_decision_unix":1779779954.910028,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":127488,"affinity_cache_ratio":0.9916383407356705,"affinity_num_requests":2,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":127488,"estimated_new_tokens":1075,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":55818,"ongoing_decode_tokens":55818,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1660,"cache_hit":127488,"new_prefill":1075,"score_linear":-71670.0,"score_lmetric":2150},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1892,"cache_hit":1024,"new_prefill":127539,"score_linear":-1024.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2249,"cache_hit":0,"new_prefill":128563,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2157,"cache_hit":0,"new_prefill":128563,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":81299,"ongoing_decode_tokens":0,"pending_prefill_tokens":8083,"num_requests":1,"active_p_offloads":0,"cached_blocks":2720,"cache_hit":6656,"new_prefill":121907,"score_linear":74643.0,"score_lmetric":129990},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1742,"cache_hit":0,"new_prefill":128563,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":159107,"ongoing_decode_tokens":0,"pending_prefill_tokens":65411,"num_requests":2,"active_p_offloads":0,"cached_blocks":1588,"cache_hit":0,"new_prefill":128563,"score_linear":159107.0,"score_lmetric":387948},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1688,"cache_hit":0,"new_prefill":128563,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-71670.0,"chosen_score_lmetric":2150,"t_first_token":349090.974738195,"t_first_token_unix":1779779955.988682,"t_done":349111.466545086,"t_done_unix":1779779976.4804888},{"request_id":"1405155:3:1469167:897","session_id":"1405155","input_length":66246,"t_proxy_recv":349113.009884568,"t_decision_unix":1779779978.023826,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":40448,"affinity_cache_ratio":0.610572713824231,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":40448,"estimated_new_tokens":25798,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1685,"cache_hit":0,"new_prefill":66246,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1894,"cache_hit":0,"new_prefill":66246,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2249,"cache_hit":0,"new_prefill":66246,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2157,"cache_hit":40448,"new_prefill":25798,"score_linear":-40448.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":0,"new_prefill":66246,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1747,"cache_hit":512,"new_prefill":65734,"score_linear":-512.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":164602,"ongoing_decode_tokens":34671,"pending_prefill_tokens":64907,"num_requests":2,"active_p_offloads":0,"cached_blocks":1591,"cache_hit":0,"new_prefill":66246,"score_linear":164602.0,"score_lmetric":262306},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1688,"cache_hit":0,"new_prefill":66246,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-40448.0,"chosen_score_lmetric":0,"t_first_token":349121.921237941,"t_first_token_unix":1779779986.9351814,"t_done":349123.752705389,"t_done_unix":1779779988.7666488},{"request_id":"1399948:13:1472968:898","session_id":"1399948","input_length":12247,"t_proxy_recv":349125.463702079,"t_decision_unix":1779779990.4776433,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":11776,"affinity_cache_ratio":0.9615416020249857,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":11776,"estimated_new_tokens":471,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1685,"cache_hit":0,"new_prefill":12247,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1894,"cache_hit":0,"new_prefill":12247,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2249,"cache_hit":11776,"new_prefill":471,"score_linear":-11776.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2207,"cache_hit":0,"new_prefill":12247,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":0,"new_prefill":12247,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1747,"cache_hit":0,"new_prefill":12247,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":164602,"ongoing_decode_tokens":34671,"pending_prefill_tokens":64907,"num_requests":2,"active_p_offloads":0,"cached_blocks":1591,"cache_hit":0,"new_prefill":12247,"score_linear":164602.0,"score_lmetric":154308},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1688,"cache_hit":6144,"new_prefill":6103,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-11776.0,"chosen_score_lmetric":0,"t_first_token":349125.516841959,"t_first_token_unix":1779779990.5307853,"t_done":349125.655330194,"t_done_unix":1779779990.6692731},{"request_id":"1342921:7:1473812:900","session_id":"1342921","input_length":76600,"t_proxy_recv":349128.428770127,"t_decision_unix":1779779993.4427102,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":75776,"affinity_cache_ratio":0.9892428198433421,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":75776,"estimated_new_tokens":824,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1685,"cache_hit":0,"new_prefill":76600,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1894,"cache_hit":0,"new_prefill":76600,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2249,"cache_hit":0,"new_prefill":76600,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2207,"cache_hit":75776,"new_prefill":824,"score_linear":-75776.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":0,"new_prefill":76600,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":24569,"ongoing_decode_tokens":24569,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1747,"cache_hit":0,"new_prefill":76600,"score_linear":24569.0,"score_lmetric":76600},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":164602,"ongoing_decode_tokens":34671,"pending_prefill_tokens":64907,"num_requests":2,"active_p_offloads":0,"cached_blocks":1591,"cache_hit":0,"new_prefill":76600,"score_linear":164602.0,"score_lmetric":283014},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1688,"cache_hit":0,"new_prefill":76600,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-75776.0,"chosen_score_lmetric":0,"t_first_token":349128.963292575,"t_first_token_unix":1779779993.977236,"t_done":349130.303966702,"t_done_unix":1779779995.3179097},{"request_id":"1371622:8:1473416:899","session_id":"1371622","input_length":24569,"t_proxy_recv":349127.131836803,"t_decision_unix":1779779992.1457777,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":24064,"affinity_cache_ratio":0.9794456428833083,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":24064,"estimated_new_tokens":505,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1685,"cache_hit":0,"new_prefill":24569,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1894,"cache_hit":0,"new_prefill":24569,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2249,"cache_hit":0,"new_prefill":24569,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2207,"cache_hit":0,"new_prefill":24569,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":11264,"new_prefill":13305,"score_linear":-11264.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1747,"cache_hit":24064,"new_prefill":505,"score_linear":-24064.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":164602,"ongoing_decode_tokens":34671,"pending_prefill_tokens":64907,"num_requests":2,"active_p_offloads":0,"cached_blocks":1591,"cache_hit":0,"new_prefill":24569,"score_linear":164602.0,"score_lmetric":178952},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1688,"cache_hit":0,"new_prefill":24569,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-24064.0,"chosen_score_lmetric":0,"t_first_token":349127.289518381,"t_first_token_unix":1779779992.3034618,"t_done":349130.446674438,"t_done_unix":1779779995.4606178},{"request_id":"1363943:16:1463626:890","session_id":"1363943","input_length":34671,"t_proxy_recv":349105.15295046,"t_decision_unix":1779779970.166892,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":29696,"affinity_cache_ratio":0.8565083210752502,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":29696,"estimated_new_tokens":4975,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":162955,"ongoing_decode_tokens":162955,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1675,"cache_hit":8704,"new_prefill":25967,"score_linear":154251.0,"score_lmetric":51934},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1894,"cache_hit":0,"new_prefill":34671,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2249,"cache_hit":0,"new_prefill":34671,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2157,"cache_hit":0,"new_prefill":34671,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":4096,"new_prefill":30575,"score_linear":-4096.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1745,"cache_hit":0,"new_prefill":34671,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":64724,"ongoing_decode_tokens":64724,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1590,"cache_hit":29696,"new_prefill":4975,"score_linear":35028.0,"score_lmetric":4975},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1688,"cache_hit":4096,"new_prefill":30575,"score_linear":-4096.0,"score_lmetric":0}],"chosen_score_linear":35028.0,"chosen_score_lmetric":4975,"t_first_token":349106.303741547,"t_first_token_unix":1779779971.317685,"t_done":349147.4508309,"t_done_unix":1779780012.4647746},{"request_id":"1305906:4:1476757:902","session_id":"1305906","input_length":44949,"t_proxy_recv":349138.495369174,"t_decision_unix":1779780003.5093102,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":16896,"affinity_cache_ratio":0.375892678368818,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":16896,"estimated_new_tokens":28053,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1685,"cache_hit":0,"new_prefill":44949,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1894,"cache_hit":0,"new_prefill":44949,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2249,"cache_hit":16896,"new_prefill":28053,"score_linear":-16896.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2208,"cache_hit":0,"new_prefill":44949,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":0,"new_prefill":44949,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1748,"cache_hit":0,"new_prefill":44949,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":164602,"ongoing_decode_tokens":34671,"pending_prefill_tokens":64907,"num_requests":2,"active_p_offloads":0,"cached_blocks":1591,"cache_hit":0,"new_prefill":44949,"score_linear":164602.0,"score_lmetric":219712},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1688,"cache_hit":0,"new_prefill":44949,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-16896.0,"chosen_score_lmetric":0,"t_first_token":349146.296649367,"t_first_token_unix":1779780011.3105927,"t_done":349147.638047274,"t_done_unix":1779780012.6519902},{"request_id":"1313181:11:1344301:464","session_id":"1313181","input_length":129931,"t_proxy_recv":349107.92503917,"t_decision_unix":1779779972.9389808,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":65024,"affinity_cache_ratio":0.5004502389729933,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":65024,"estimated_new_tokens":64907,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":163426,"ongoing_decode_tokens":163426,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1680,"cache_hit":24064,"new_prefill":105867,"score_linear":139362.0,"score_lmetric":211734},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1894,"cache_hit":0,"new_prefill":129931,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2249,"cache_hit":0,"new_prefill":129931,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2157,"cache_hit":0,"new_prefill":129931,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":0,"new_prefill":129931,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1745,"cache_hit":0,"new_prefill":129931,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":34671,"ongoing_decode_tokens":34671,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1591,"cache_hit":65024,"new_prefill":64907,"score_linear":-30353.0,"score_lmetric":64907},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1688,"cache_hit":62464,"new_prefill":67467,"score_linear":-62464.0,"score_lmetric":0}],"chosen_score_linear":-30353.0,"chosen_score_lmetric":64907,"t_first_token":349146.698246923,"t_first_token_unix":1779780011.7121902,"t_done":349148.05503387,"t_done_unix":1779780013.068977},{"request_id":"1371622:9:1479535:905","session_id":"1371622","input_length":25286,"t_proxy_recv":349147.809943765,"t_decision_unix":1779780012.8238852,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":24576,"affinity_cache_ratio":0.9719212212291386,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":24576,"estimated_new_tokens":710,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":38827,"ongoing_decode_tokens":0,"pending_prefill_tokens":427,"num_requests":1,"active_p_offloads":0,"cached_blocks":1685,"cache_hit":0,"new_prefill":25286,"score_linear":38827.0,"score_lmetric":25713},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1894,"cache_hit":0,"new_prefill":25286,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2304,"cache_hit":0,"new_prefill":25286,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2208,"cache_hit":0,"new_prefill":25286,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":11264,"new_prefill":14022,"score_linear":-11264.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1748,"cache_hit":24576,"new_prefill":710,"score_linear":-24576.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":165922,"ongoing_decode_tokens":129931,"pending_prefill_tokens":1175,"num_requests":2,"active_p_offloads":0,"cached_blocks":1601,"cache_hit":0,"new_prefill":25286,"score_linear":165922.0,"score_lmetric":52922},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1688,"cache_hit":0,"new_prefill":25286,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-24576.0,"chosen_score_lmetric":0,"t_first_token":349147.924393702,"t_first_token_unix":1779780012.9383376,"t_done":349148.474133766,"t_done_unix":1779780013.4880772},{"request_id":"1313181:11:1402841:762","session_id":"1313181","input_length":65123,"t_proxy_recv":349148.069270331,"t_decision_unix":1779780013.0832121,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":65024,"affinity_cache_ratio":0.9984797997635244,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":65024,"estimated_new_tokens":99,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":38827,"ongoing_decode_tokens":0,"pending_prefill_tokens":427,"num_requests":1,"active_p_offloads":0,"cached_blocks":1685,"cache_hit":24064,"new_prefill":41059,"score_linear":14763.0,"score_lmetric":41486},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1894,"cache_hit":0,"new_prefill":65123,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2304,"cache_hit":0,"new_prefill":65123,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2208,"cache_hit":0,"new_prefill":65123,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":0,"new_prefill":65123,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":25286,"ongoing_decode_tokens":25286,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1748,"cache_hit":0,"new_prefill":65123,"score_linear":25286.0,"score_lmetric":65123},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":35991,"ongoing_decode_tokens":35991,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1602,"cache_hit":65024,"new_prefill":99,"score_linear":-29033.0,"score_lmetric":99},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1688,"cache_hit":62464,"new_prefill":2659,"score_linear":-62464.0,"score_lmetric":0}],"chosen_score_linear":-29033.0,"chosen_score_lmetric":99,"t_first_token":349148.15827753,"t_first_token_unix":1779780013.172221,"t_done":349148.690408509,"t_done_unix":1779780013.7043521},{"request_id":"1363943:17:1466811:893","session_id":"1363943","input_length":35991,"t_proxy_recv":349147.461434872,"t_decision_unix":1779780012.4753764,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":34816,"affinity_cache_ratio":0.9673529493484482,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":34816,"estimated_new_tokens":1175,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":38827,"ongoing_decode_tokens":0,"pending_prefill_tokens":427,"num_requests":1,"active_p_offloads":0,"cached_blocks":1685,"cache_hit":8704,"new_prefill":27287,"score_linear":30123.0,"score_lmetric":27714},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1894,"cache_hit":0,"new_prefill":35991,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":44949,"ongoing_decode_tokens":44949,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2249,"cache_hit":0,"new_prefill":35991,"score_linear":44949.0,"score_lmetric":35991},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2208,"cache_hit":0,"new_prefill":35991,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":4096,"new_prefill":31895,"score_linear":-4096.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1748,"cache_hit":0,"new_prefill":35991,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":129931,"ongoing_decode_tokens":129931,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1601,"cache_hit":34816,"new_prefill":1175,"score_linear":95115.0,"score_lmetric":1175},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1688,"cache_hit":4096,"new_prefill":31895,"score_linear":-4096.0,"score_lmetric":0}],"chosen_score_linear":95115.0,"chosen_score_lmetric":1175,"t_first_token":349147.835261536,"t_first_token_unix":1779780012.8492048,"t_done":349149.313873493,"t_done_unix":1779780014.3278167},{"request_id":"1363943:18:1469143:896","session_id":"1363943","input_length":47303,"t_proxy_recv":349149.327378964,"t_decision_unix":1779780014.3413208,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":35840,"affinity_cache_ratio":0.7576686468088705,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":35840,"estimated_new_tokens":11463,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":38827,"ongoing_decode_tokens":0,"pending_prefill_tokens":427,"num_requests":1,"active_p_offloads":0,"cached_blocks":1685,"cache_hit":8704,"new_prefill":38599,"score_linear":30123.0,"score_lmetric":39026},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1894,"cache_hit":0,"new_prefill":47303,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2304,"cache_hit":0,"new_prefill":47303,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2208,"cache_hit":0,"new_prefill":47303,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":4096,"new_prefill":43207,"score_linear":-4096.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1749,"cache_hit":0,"new_prefill":47303,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1604,"cache_hit":35840,"new_prefill":11463,"score_linear":-35840.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":130479,"ongoing_decode_tokens":0,"pending_prefill_tokens":68015,"num_requests":1,"active_p_offloads":0,"cached_blocks":1688,"cache_hit":4096,"new_prefill":43207,"score_linear":126383.0,"score_lmetric":111222}],"chosen_score_linear":-35840.0,"chosen_score_lmetric":0,"t_first_token":349152.535793859,"t_first_token_unix":1779780017.5497375,"t_done":349155.284343912,"t_done_unix":1779780020.2982874},{"request_id":"1404395:5:1480170:906","session_id":"1404395","input_length":36988,"t_proxy_recv":349150.049953717,"t_decision_unix":1779780015.0638952,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":34816,"affinity_cache_ratio":0.9412782524061858,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":34816,"estimated_new_tokens":2172,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":38827,"ongoing_decode_tokens":0,"pending_prefill_tokens":427,"num_requests":1,"active_p_offloads":0,"cached_blocks":1685,"cache_hit":34816,"new_prefill":2172,"score_linear":4011.0,"score_lmetric":2599},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1894,"cache_hit":0,"new_prefill":36988,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2304,"cache_hit":0,"new_prefill":36988,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2208,"cache_hit":0,"new_prefill":36988,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":0,"new_prefill":36988,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1749,"cache_hit":512,"new_prefill":36476,"score_linear":-512.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":47303,"ongoing_decode_tokens":0,"pending_prefill_tokens":11463,"num_requests":1,"active_p_offloads":0,"cached_blocks":1604,"cache_hit":0,"new_prefill":36988,"score_linear":47303.0,"score_lmetric":48451},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":130479,"ongoing_decode_tokens":0,"pending_prefill_tokens":68015,"num_requests":1,"active_p_offloads":0,"cached_blocks":1688,"cache_hit":0,"new_prefill":36988,"score_linear":130479.0,"score_lmetric":105003}],"chosen_score_linear":4011.0,"chosen_score_lmetric":2599,"t_first_token":349152.424467409,"t_first_token_unix":1779780017.4384108,"t_done":349156.102108539,"t_done_unix":1779780021.1160514},{"request_id":"1268861:13:1481831:907","session_id":"1268861","input_length":57130,"t_proxy_recv":349155.641487959,"t_decision_unix":1779780020.6554277,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":55808,"affinity_cache_ratio":0.976859793453527,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":55808,"estimated_new_tokens":1322,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":75815,"ongoing_decode_tokens":75815,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1685,"cache_hit":0,"new_prefill":57130,"score_linear":75815.0,"score_lmetric":114260},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1894,"cache_hit":55808,"new_prefill":1322,"score_linear":-55808.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2304,"cache_hit":0,"new_prefill":57130,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2208,"cache_hit":0,"new_prefill":57130,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":0,"new_prefill":57130,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1749,"cache_hit":0,"new_prefill":57130,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":56056,"ongoing_decode_tokens":0,"pending_prefill_tokens":8440,"num_requests":1,"active_p_offloads":0,"cached_blocks":1627,"cache_hit":0,"new_prefill":57130,"score_linear":56056.0,"score_lmetric":65570},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":130479,"ongoing_decode_tokens":0,"pending_prefill_tokens":68015,"num_requests":1,"active_p_offloads":0,"cached_blocks":1688,"cache_hit":12288,"new_prefill":44842,"score_linear":118191.0,"score_lmetric":112857}],"chosen_score_linear":-55808.0,"chosen_score_lmetric":0,"t_first_token":349156.068847283,"t_first_token_unix":1779780021.0827904,"t_done":349157.368545953,"t_done_unix":1779780022.3824892},{"request_id":"1363943:19:1473899:901","session_id":"1363943","input_length":56056,"t_proxy_recv":349155.298349096,"t_decision_unix":1779780020.312291,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":47616,"affinity_cache_ratio":0.8494362780077066,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":47616,"estimated_new_tokens":8440,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":75815,"ongoing_decode_tokens":75815,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1685,"cache_hit":8704,"new_prefill":47352,"score_linear":67111.0,"score_lmetric":94704},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1894,"cache_hit":0,"new_prefill":56056,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2304,"cache_hit":0,"new_prefill":56056,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2208,"cache_hit":0,"new_prefill":56056,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":4096,"new_prefill":51960,"score_linear":-4096.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1749,"cache_hit":0,"new_prefill":56056,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1627,"cache_hit":47616,"new_prefill":8440,"score_linear":-47616.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":130479,"ongoing_decode_tokens":0,"pending_prefill_tokens":68015,"num_requests":1,"active_p_offloads":0,"cached_blocks":1688,"cache_hit":4096,"new_prefill":51960,"score_linear":126383.0,"score_lmetric":119975}],"chosen_score_linear":-47616.0,"chosen_score_lmetric":0,"t_first_token":349158.256472072,"t_first_token_unix":1779780023.2704153,"t_done":349160.101163113,"t_done_unix":1779780025.1151063},{"request_id":"1363943:20:1476805:903","session_id":"1363943","input_length":60761,"t_proxy_recv":349160.119855395,"t_decision_unix":1779780025.1337967,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":55808,"affinity_cache_ratio":0.9184838959200803,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":55808,"estimated_new_tokens":4953,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":38827,"ongoing_decode_tokens":38827,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1689,"cache_hit":8704,"new_prefill":52057,"score_linear":30123.0,"score_lmetric":52057},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1896,"cache_hit":0,"new_prefill":60761,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2304,"cache_hit":0,"new_prefill":60761,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2208,"cache_hit":0,"new_prefill":60761,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":4096,"new_prefill":56665,"score_linear":-4096.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1749,"cache_hit":0,"new_prefill":60761,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1643,"cache_hit":55808,"new_prefill":4953,"score_linear":-55808.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":130479,"ongoing_decode_tokens":0,"pending_prefill_tokens":68015,"num_requests":1,"active_p_offloads":0,"cached_blocks":1688,"cache_hit":4096,"new_prefill":56665,"score_linear":126383.0,"score_lmetric":124680}],"chosen_score_linear":-55808.0,"chosen_score_lmetric":0,"t_first_token":349161.903273496,"t_first_token_unix":1779780026.917217,"t_done":349165.360764907,"t_done_unix":1779780030.374708},{"request_id":"1301929:4:1479037:904","session_id":"1301929","input_length":38827,"t_proxy_recv":349146.070729065,"t_decision_unix":1779780011.0846703,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":38400,"affinity_cache_ratio":0.9890024982615191,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":38400,"estimated_new_tokens":427,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1685,"cache_hit":38400,"new_prefill":427,"score_linear":-38400.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1894,"cache_hit":0,"new_prefill":38827,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":44949,"ongoing_decode_tokens":0,"pending_prefill_tokens":28053,"num_requests":1,"active_p_offloads":0,"cached_blocks":2249,"cache_hit":0,"new_prefill":38827,"score_linear":44949.0,"score_lmetric":66880},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2208,"cache_hit":0,"new_prefill":38827,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":0,"new_prefill":38827,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1748,"cache_hit":0,"new_prefill":38827,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":164602,"ongoing_decode_tokens":34671,"pending_prefill_tokens":64907,"num_requests":2,"active_p_offloads":0,"cached_blocks":1591,"cache_hit":0,"new_prefill":38827,"score_linear":164602.0,"score_lmetric":207468},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1688,"cache_hit":0,"new_prefill":38827,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-38400.0,"chosen_score_lmetric":0,"t_first_token":349151.884021795,"t_first_token_unix":1779780016.8979657,"t_done":349165.925363066,"t_done_unix":1779780030.9393065},{"request_id":"1404395:6:1483967:909","session_id":"1404395","input_length":37689,"t_proxy_recv":349162.891233491,"t_decision_unix":1779780027.9051747,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":36864,"affinity_cache_ratio":0.9781103239672053,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":36864,"estimated_new_tokens":825,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":38827,"ongoing_decode_tokens":38827,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1689,"cache_hit":36864,"new_prefill":825,"score_linear":1963.0,"score_lmetric":825},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1896,"cache_hit":0,"new_prefill":37689,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2304,"cache_hit":0,"new_prefill":37689,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2208,"cache_hit":0,"new_prefill":37689,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":0,"new_prefill":37689,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1749,"cache_hit":512,"new_prefill":37177,"score_linear":-512.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":60761,"ongoing_decode_tokens":60761,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1643,"cache_hit":0,"new_prefill":37689,"score_linear":60761.0,"score_lmetric":37689},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":130479,"ongoing_decode_tokens":0,"pending_prefill_tokens":68015,"num_requests":1,"active_p_offloads":0,"cached_blocks":1688,"cache_hit":0,"new_prefill":37689,"score_linear":130479.0,"score_lmetric":105704}],"chosen_score_linear":1963.0,"chosen_score_lmetric":825,"t_first_token":349163.097912179,"t_first_token_unix":1779780028.111856,"t_done":349166.429678051,"t_done_unix":1779780031.4436214},{"request_id":"1363943:21:1482184:908","session_id":"1363943","input_length":65658,"t_proxy_recv":349165.374702017,"t_decision_unix":1779780030.3886435,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":60928,"affinity_cache_ratio":0.9279600353346127,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":60928,"estimated_new_tokens":4730,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":76516,"ongoing_decode_tokens":76516,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1689,"cache_hit":8704,"new_prefill":56954,"score_linear":67812.0,"score_lmetric":113908},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1896,"cache_hit":0,"new_prefill":65658,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2304,"cache_hit":0,"new_prefill":65658,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2208,"cache_hit":0,"new_prefill":65658,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":4096,"new_prefill":61562,"score_linear":-4096.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1749,"cache_hit":0,"new_prefill":65658,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1653,"cache_hit":60928,"new_prefill":4730,"score_linear":-60928.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":130479,"ongoing_decode_tokens":0,"pending_prefill_tokens":68015,"num_requests":1,"active_p_offloads":0,"cached_blocks":1688,"cache_hit":4096,"new_prefill":61562,"score_linear":126383.0,"score_lmetric":129577}],"chosen_score_linear":-60928.0,"chosen_score_lmetric":0,"t_first_token":349167.28969603,"t_first_token_unix":1779780032.30364,"t_done":349170.428894729,"t_done_unix":1779780035.4428387},{"request_id":"1242838:6:1486863:911","session_id":"1242838","input_length":29903,"t_proxy_recv":349172.644058785,"t_decision_unix":1779780037.6580002,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":26624,"affinity_cache_ratio":0.8903454502892686,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":26624,"estimated_new_tokens":3279,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1691,"cache_hit":26624,"new_prefill":3279,"score_linear":-26624.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1896,"cache_hit":0,"new_prefill":29903,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2304,"cache_hit":0,"new_prefill":29903,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2208,"cache_hit":0,"new_prefill":29903,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":0,"new_prefill":29903,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1749,"cache_hit":0,"new_prefill":29903,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":70238,"ongoing_decode_tokens":70238,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1662,"cache_hit":0,"new_prefill":29903,"score_linear":70238.0,"score_lmetric":29903},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":130479,"ongoing_decode_tokens":0,"pending_prefill_tokens":68015,"num_requests":1,"active_p_offloads":0,"cached_blocks":1688,"cache_hit":0,"new_prefill":29903,"score_linear":130479.0,"score_lmetric":97918}],"chosen_score_linear":-26624.0,"chosen_score_lmetric":0,"t_first_token":349173.370312105,"t_first_token_unix":1779780038.384256,"t_done":349174.725896866,"t_done_unix":1779780039.73984},{"request_id":"1363943:22:1485341:910","session_id":"1363943","input_length":70238,"t_proxy_recv":349170.44383242,"t_decision_unix":1779780035.457774,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":65536,"affinity_cache_ratio":0.9330561804151599,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":65536,"estimated_new_tokens":4702,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1691,"cache_hit":8704,"new_prefill":61534,"score_linear":-8704.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1896,"cache_hit":0,"new_prefill":70238,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2304,"cache_hit":0,"new_prefill":70238,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2208,"cache_hit":0,"new_prefill":70238,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":4096,"new_prefill":66142,"score_linear":-4096.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1749,"cache_hit":0,"new_prefill":70238,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1662,"cache_hit":65536,"new_prefill":4702,"score_linear":-65536.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":130479,"ongoing_decode_tokens":0,"pending_prefill_tokens":68015,"num_requests":1,"active_p_offloads":0,"cached_blocks":1688,"cache_hit":4096,"new_prefill":66142,"score_linear":126383.0,"score_lmetric":134157}],"chosen_score_linear":-65536.0,"chosen_score_lmetric":0,"t_first_token":349172.356300264,"t_first_token_unix":1779780037.3702443,"t_done":349175.837240552,"t_done_unix":1779780040.8511846},{"request_id":"1363943:23:1488713:912","session_id":"1363943","input_length":72516,"t_proxy_recv":349178.779936446,"t_decision_unix":1779780043.7938778,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":70144,"affinity_cache_ratio":0.9672899773843014,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":70144,"estimated_new_tokens":2372,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1697,"cache_hit":8704,"new_prefill":63812,"score_linear":-8704.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1896,"cache_hit":0,"new_prefill":72516,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2304,"cache_hit":0,"new_prefill":72516,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2208,"cache_hit":0,"new_prefill":72516,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":4096,"new_prefill":68420,"score_linear":-4096.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1749,"cache_hit":0,"new_prefill":72516,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1671,"cache_hit":70144,"new_prefill":2372,"score_linear":-70144.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":130479,"ongoing_decode_tokens":0,"pending_prefill_tokens":68015,"num_requests":1,"active_p_offloads":0,"cached_blocks":1688,"cache_hit":4096,"new_prefill":68420,"score_linear":126383.0,"score_lmetric":136435}],"chosen_score_linear":-70144.0,"chosen_score_lmetric":0,"t_first_token":349179.82386028,"t_first_token_unix":1779780044.837804,"t_done":349181.801292778,"t_done_unix":1779780046.8152363},{"request_id":"1404395:7:1491186:914","session_id":"1404395","input_length":39569,"t_proxy_recv":349187.438089122,"t_decision_unix":1779780052.4520304,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":37888,"affinity_cache_ratio":0.9575172483509818,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":37888,"estimated_new_tokens":1681,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1697,"cache_hit":37888,"new_prefill":1681,"score_linear":-37888.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1896,"cache_hit":0,"new_prefill":39569,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2304,"cache_hit":0,"new_prefill":39569,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2208,"cache_hit":0,"new_prefill":39569,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":0,"new_prefill":39569,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1749,"cache_hit":512,"new_prefill":39057,"score_linear":-512.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":74939,"ongoing_decode_tokens":0,"pending_prefill_tokens":2235,"num_requests":1,"active_p_offloads":0,"cached_blocks":1676,"cache_hit":0,"new_prefill":39569,"score_linear":74939.0,"score_lmetric":41804},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":130479,"ongoing_decode_tokens":0,"pending_prefill_tokens":68015,"num_requests":1,"active_p_offloads":0,"cached_blocks":1688,"cache_hit":0,"new_prefill":39569,"score_linear":130479.0,"score_lmetric":107584}],"chosen_score_linear":-37888.0,"chosen_score_lmetric":0,"t_first_token":349187.930191662,"t_first_token_unix":1779780052.944135,"t_done":349188.211777189,"t_done_unix":1779780053.2257204},{"request_id":"1313181:12:1346194:470","session_id":"1313181","input_length":130479,"t_proxy_recv":349148.718239355,"t_decision_unix":1779780013.7321808,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":6,"chosen_idx":7,"affinity_cache_hit":65024,"affinity_cache_ratio":0.49834839322802904,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":62464,"estimated_new_tokens":68015,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":38827,"ongoing_decode_tokens":0,"pending_prefill_tokens":427,"num_requests":1,"active_p_offloads":0,"cached_blocks":1685,"cache_hit":24064,"new_prefill":106415,"score_linear":14763.0,"score_lmetric":106842},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1894,"cache_hit":0,"new_prefill":130479,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2304,"cache_hit":0,"new_prefill":130479,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2208,"cache_hit":0,"new_prefill":130479,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":0,"new_prefill":130479,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1749,"cache_hit":0,"new_prefill":130479,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":35991,"ongoing_decode_tokens":35991,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1602,"cache_hit":65024,"new_prefill":65455,"score_linear":-29033.0,"score_lmetric":65455},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1688,"cache_hit":62464,"new_prefill":68015,"score_linear":-62464.0,"score_lmetric":0}],"chosen_score_linear":-62464.0,"chosen_score_lmetric":0,"t_first_token":349188.776132768,"t_first_token_unix":1779780053.790076,"t_done":349189.686486363,"t_done_unix":1779780054.7004297},{"request_id":"1363943:24:1491020:913","session_id":"1363943","input_length":74939,"t_proxy_recv":349187.042140389,"t_decision_unix":1779780052.0560818,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":72704,"affinity_cache_ratio":0.9701757429375892,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":72704,"estimated_new_tokens":2235,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1697,"cache_hit":8704,"new_prefill":66235,"score_linear":-8704.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1896,"cache_hit":0,"new_prefill":74939,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2304,"cache_hit":0,"new_prefill":74939,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2208,"cache_hit":0,"new_prefill":74939,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":4096,"new_prefill":70843,"score_linear":-4096.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1749,"cache_hit":0,"new_prefill":74939,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1676,"cache_hit":72704,"new_prefill":2235,"score_linear":-72704.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":130479,"ongoing_decode_tokens":0,"pending_prefill_tokens":68015,"num_requests":1,"active_p_offloads":0,"cached_blocks":1688,"cache_hit":4096,"new_prefill":70843,"score_linear":126383.0,"score_lmetric":138858}],"chosen_score_linear":-72704.0,"chosen_score_lmetric":0,"t_first_token":349188.256992417,"t_first_token_unix":1779780053.2709355,"t_done":349189.854576753,"t_done_unix":1779780054.86852},{"request_id":"1313181:12:1404813:772","session_id":"1313181","input_length":65179,"t_proxy_recv":349189.697102844,"t_decision_unix":1779780054.7110443,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":65024,"affinity_cache_ratio":0.9976219334448212,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":65024,"estimated_new_tokens":155,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1700,"cache_hit":24064,"new_prefill":41115,"score_linear":-24064.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1896,"cache_hit":0,"new_prefill":65179,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2304,"cache_hit":0,"new_prefill":65179,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2208,"cache_hit":0,"new_prefill":65179,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":0,"new_prefill":65179,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1749,"cache_hit":0,"new_prefill":65179,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":74939,"ongoing_decode_tokens":74939,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1676,"cache_hit":65024,"new_prefill":155,"score_linear":9915.0,"score_lmetric":155},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1707,"cache_hit":65024,"new_prefill":155,"score_linear":-65024.0,"score_lmetric":0}],"chosen_score_linear":-65024.0,"chosen_score_lmetric":0,"t_first_token":349189.819630528,"t_first_token_unix":1779780054.833574,"t_done":349190.353060335,"t_done_unix":1779780055.367004},{"request_id":"1363943:25:1493325:915","session_id":"1363943","input_length":75711,"t_proxy_recv":349194.528339801,"t_decision_unix":1779780059.542281,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":74752,"affinity_cache_ratio":0.9873334125820554,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":74752,"estimated_new_tokens":959,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1700,"cache_hit":8704,"new_prefill":67007,"score_linear":-8704.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1896,"cache_hit":0,"new_prefill":75711,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2304,"cache_hit":0,"new_prefill":75711,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2208,"cache_hit":0,"new_prefill":75711,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":4096,"new_prefill":71615,"score_linear":-4096.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1749,"cache_hit":0,"new_prefill":75711,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1680,"cache_hit":74752,"new_prefill":959,"score_linear":-74752.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":131699,"ongoing_decode_tokens":0,"pending_prefill_tokens":66675,"num_requests":1,"active_p_offloads":0,"cached_blocks":1707,"cache_hit":4096,"new_prefill":71615,"score_linear":127603.0,"score_lmetric":138290}],"chosen_score_linear":-74752.0,"chosen_score_lmetric":0,"t_first_token":349194.92453923,"t_first_token_unix":1779780059.938483,"t_done":349197.1183517,"t_done_unix":1779780062.1322954},{"request_id":"1404395:8:1494219:916","session_id":"1404395","input_length":39820,"t_proxy_recv":349197.545594361,"t_decision_unix":1779780062.559536,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":39424,"affinity_cache_ratio":0.9900552486187846,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":39424,"estimated_new_tokens":396,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1700,"cache_hit":39424,"new_prefill":396,"score_linear":-39424.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1896,"cache_hit":0,"new_prefill":39820,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2304,"cache_hit":0,"new_prefill":39820,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2208,"cache_hit":0,"new_prefill":39820,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":0,"new_prefill":39820,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1749,"cache_hit":512,"new_prefill":39308,"score_linear":-512.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1682,"cache_hit":0,"new_prefill":39820,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":131699,"ongoing_decode_tokens":0,"pending_prefill_tokens":66675,"num_requests":1,"active_p_offloads":0,"cached_blocks":1707,"cache_hit":0,"new_prefill":39820,"score_linear":131699.0,"score_lmetric":106495}],"chosen_score_linear":-39424.0,"chosen_score_lmetric":0,"t_first_token":349197.692150641,"t_first_token_unix":1779780062.7060943,"t_done":349201.321071294,"t_done_unix":1779780066.3350163},{"request_id":"1363943:26:1495648:917","session_id":"1363943","input_length":83861,"t_proxy_recv":349202.288852119,"t_decision_unix":1779780067.3027935,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":75776,"affinity_cache_ratio":0.903590465174515,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":75776,"estimated_new_tokens":8085,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1701,"cache_hit":8704,"new_prefill":75157,"score_linear":-8704.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1896,"cache_hit":0,"new_prefill":83861,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2304,"cache_hit":0,"new_prefill":83861,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2208,"cache_hit":0,"new_prefill":83861,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":4096,"new_prefill":79765,"score_linear":-4096.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1749,"cache_hit":0,"new_prefill":83861,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1682,"cache_hit":75776,"new_prefill":8085,"score_linear":-75776.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":131699,"ongoing_decode_tokens":0,"pending_prefill_tokens":66675,"num_requests":1,"active_p_offloads":0,"cached_blocks":1707,"cache_hit":4096,"new_prefill":79765,"score_linear":127603.0,"score_lmetric":146440}],"chosen_score_linear":-75776.0,"chosen_score_lmetric":0,"t_first_token":349206.315124351,"t_first_token_unix":1779780071.3290682,"t_done":349208.15569436,"t_done_unix":1779780073.1696382},{"request_id":"1363943:27:1497882:919","session_id":"1363943","input_length":84090,"t_proxy_recv":349209.558825365,"t_decision_unix":1779780074.5727665,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":83968,"affinity_cache_ratio":0.9985491735045784,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":83968,"estimated_new_tokens":122,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1701,"cache_hit":8704,"new_prefill":75386,"score_linear":-8704.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":75760,"ongoing_decode_tokens":0,"pending_prefill_tokens":4080,"num_requests":1,"active_p_offloads":0,"cached_blocks":1896,"cache_hit":0,"new_prefill":84090,"score_linear":75760.0,"score_lmetric":88170},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2304,"cache_hit":0,"new_prefill":84090,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2208,"cache_hit":0,"new_prefill":84090,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":4096,"new_prefill":79994,"score_linear":-4096.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1749,"cache_hit":0,"new_prefill":84090,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1698,"cache_hit":83968,"new_prefill":122,"score_linear":-83968.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":131699,"ongoing_decode_tokens":0,"pending_prefill_tokens":66675,"num_requests":1,"active_p_offloads":0,"cached_blocks":1707,"cache_hit":4096,"new_prefill":79994,"score_linear":127603.0,"score_lmetric":146669}],"chosen_score_linear":-83968.0,"chosen_score_lmetric":0,"t_first_token":349209.691233413,"t_first_token_unix":1779780074.7051766,"t_done":349213.258405292,"t_done_unix":1779780078.272349},{"request_id":"1242838:7:1499476:921","session_id":"1242838","input_length":33352,"t_proxy_recv":349214.870522019,"t_decision_unix":1779780079.884463,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":29696,"affinity_cache_ratio":0.8903813864236028,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":29696,"estimated_new_tokens":3656,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1701,"cache_hit":29696,"new_prefill":3656,"score_linear":-29696.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":137670,"ongoing_decode_tokens":0,"pending_prefill_tokens":9158,"num_requests":2,"active_p_offloads":0,"cached_blocks":1896,"cache_hit":0,"new_prefill":33352,"score_linear":137670.0,"score_lmetric":85020},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2304,"cache_hit":0,"new_prefill":33352,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2208,"cache_hit":0,"new_prefill":33352,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":0,"new_prefill":33352,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1749,"cache_hit":0,"new_prefill":33352,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1698,"cache_hit":0,"new_prefill":33352,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":131699,"ongoing_decode_tokens":0,"pending_prefill_tokens":66675,"num_requests":1,"active_p_offloads":0,"cached_blocks":1707,"cache_hit":0,"new_prefill":33352,"score_linear":131699.0,"score_lmetric":100027}],"chosen_score_linear":-29696.0,"chosen_score_lmetric":0,"t_first_token":349215.66922662,"t_first_token_unix":1779780080.68317,"t_done":349217.690368366,"t_done_unix":1779780082.7043116},{"request_id":"1268861:14:1498281:920","session_id":"1268861","input_length":61910,"t_proxy_recv":349211.030872043,"t_decision_unix":1779780076.044813,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":56832,"affinity_cache_ratio":0.9179777095784203,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":56832,"estimated_new_tokens":5078,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1701,"cache_hit":0,"new_prefill":61910,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":75760,"ongoing_decode_tokens":0,"pending_prefill_tokens":4080,"num_requests":1,"active_p_offloads":0,"cached_blocks":1896,"cache_hit":56832,"new_prefill":5078,"score_linear":18928.0,"score_lmetric":9158},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2304,"cache_hit":0,"new_prefill":61910,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2208,"cache_hit":0,"new_prefill":61910,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":0,"new_prefill":61910,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1749,"cache_hit":0,"new_prefill":61910,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":84090,"ongoing_decode_tokens":84090,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1698,"cache_hit":0,"new_prefill":61910,"score_linear":84090.0,"score_lmetric":61910},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":131699,"ongoing_decode_tokens":0,"pending_prefill_tokens":66675,"num_requests":1,"active_p_offloads":0,"cached_blocks":1707,"cache_hit":12288,"new_prefill":49622,"score_linear":119411.0,"score_lmetric":116297}],"chosen_score_linear":18928.0,"chosen_score_lmetric":9158,"t_first_token":349222.115111856,"t_first_token_unix":1779780087.129056,"t_done":349223.034093734,"t_done_unix":1779780088.0480375},{"request_id":"1363943:28:1500984:922","session_id":"1363943","input_length":86984,"t_proxy_recv":349219.876863183,"t_decision_unix":1779780084.8908043,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":83968,"affinity_cache_ratio":0.9653269566816886,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":83968,"estimated_new_tokens":3016,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1708,"cache_hit":8704,"new_prefill":78280,"score_linear":-8704.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":137670,"ongoing_decode_tokens":0,"pending_prefill_tokens":9158,"num_requests":2,"active_p_offloads":0,"cached_blocks":1896,"cache_hit":0,"new_prefill":86984,"score_linear":137670.0,"score_lmetric":192284},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2304,"cache_hit":0,"new_prefill":86984,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2208,"cache_hit":0,"new_prefill":86984,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":4096,"new_prefill":82888,"score_linear":-4096.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1749,"cache_hit":0,"new_prefill":86984,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1698,"cache_hit":83968,"new_prefill":3016,"score_linear":-83968.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":131699,"ongoing_decode_tokens":0,"pending_prefill_tokens":66675,"num_requests":1,"active_p_offloads":0,"cached_blocks":1707,"cache_hit":4096,"new_prefill":82888,"score_linear":127603.0,"score_lmetric":149563}],"chosen_score_linear":-83968.0,"chosen_score_lmetric":0,"t_first_token":349221.430262699,"t_first_token_unix":1779780086.444207,"t_done":349223.616566086,"t_done_unix":1779780088.6305099},{"request_id":"1342921:8:1503584:925","session_id":"1342921","input_length":78722,"t_proxy_recv":349228.724404899,"t_decision_unix":1779780093.7383463,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":76288,"affinity_cache_ratio":0.9690810700947639,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":76288,"estimated_new_tokens":2434,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1708,"cache_hit":0,"new_prefill":78722,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":179653,"ongoing_decode_tokens":75760,"pending_prefill_tokens":10709,"num_requests":2,"active_p_offloads":0,"cached_blocks":1906,"cache_hit":0,"new_prefill":78722,"score_linear":179653.0,"score_lmetric":178862},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2304,"cache_hit":0,"new_prefill":78722,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2208,"cache_hit":76288,"new_prefill":2434,"score_linear":-76288.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":0,"new_prefill":78722,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1749,"cache_hit":0,"new_prefill":78722,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":89469,"ongoing_decode_tokens":0,"pending_prefill_tokens":2429,"num_requests":1,"active_p_offloads":0,"cached_blocks":1704,"cache_hit":0,"new_prefill":78722,"score_linear":89469.0,"score_lmetric":81151},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":131699,"ongoing_decode_tokens":0,"pending_prefill_tokens":66675,"num_requests":1,"active_p_offloads":0,"cached_blocks":1707,"cache_hit":0,"new_prefill":78722,"score_linear":131699.0,"score_lmetric":145397}],"chosen_score_linear":-76288.0,"chosen_score_lmetric":0,"t_first_token":349229.869676247,"t_first_token_unix":1779780094.8836198,"t_done":349230.037424656,"t_done_unix":1779780095.0513678},{"request_id":"1388507:2:1496686:918","session_id":"1388507","input_length":75760,"t_proxy_recv":349205.649591194,"t_decision_unix":1779780070.6635325,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":71680,"affinity_cache_ratio":0.9461457233368532,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":71680,"estimated_new_tokens":4080,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1701,"cache_hit":0,"new_prefill":75760,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1896,"cache_hit":71680,"new_prefill":4080,"score_linear":-71680.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2304,"cache_hit":0,"new_prefill":75760,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2208,"cache_hit":0,"new_prefill":75760,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":0,"new_prefill":75760,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1749,"cache_hit":0,"new_prefill":75760,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":83861,"ongoing_decode_tokens":0,"pending_prefill_tokens":8085,"num_requests":1,"active_p_offloads":0,"cached_blocks":1682,"cache_hit":0,"new_prefill":75760,"score_linear":83861.0,"score_lmetric":83845},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":131699,"ongoing_decode_tokens":0,"pending_prefill_tokens":66675,"num_requests":1,"active_p_offloads":0,"cached_blocks":1707,"cache_hit":0,"new_prefill":75760,"score_linear":131699.0,"score_lmetric":142435}],"chosen_score_linear":-71680.0,"chosen_score_lmetric":0,"t_first_token":349220.478081999,"t_first_token_unix":1779780085.4920251,"t_done":349230.774802595,"t_done_unix":1779780095.7887464},{"request_id":"1363943:29:1503515:924","session_id":"1363943","input_length":89469,"t_proxy_recv":349228.470323951,"t_decision_unix":1779780093.4842653,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":87040,"affinity_cache_ratio":0.972850931607596,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":87040,"estimated_new_tokens":2429,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1708,"cache_hit":8704,"new_prefill":80765,"score_linear":-8704.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":179653,"ongoing_decode_tokens":75760,"pending_prefill_tokens":10709,"num_requests":2,"active_p_offloads":0,"cached_blocks":1906,"cache_hit":0,"new_prefill":89469,"score_linear":179653.0,"score_lmetric":200356},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2304,"cache_hit":0,"new_prefill":89469,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2208,"cache_hit":0,"new_prefill":89469,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":4096,"new_prefill":85373,"score_linear":-4096.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1749,"cache_hit":0,"new_prefill":89469,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1704,"cache_hit":87040,"new_prefill":2429,"score_linear":-87040.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":131699,"ongoing_decode_tokens":0,"pending_prefill_tokens":66675,"num_requests":1,"active_p_offloads":0,"cached_blocks":1707,"cache_hit":4096,"new_prefill":85373,"score_linear":127603.0,"score_lmetric":152048}],"chosen_score_linear":-87040.0,"chosen_score_lmetric":0,"t_first_token":349229.922869502,"t_first_token_unix":1779780094.9368129,"t_done":349231.94676537,"t_done_unix":1779780096.9607086},{"request_id":"1399948:14:1504493:926","session_id":"1399948","input_length":14189,"t_proxy_recv":349231.504340514,"t_decision_unix":1779780096.518282,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":11776,"affinity_cache_ratio":0.829938684896751,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":11776,"estimated_new_tokens":2413,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1708,"cache_hit":0,"new_prefill":14189,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":103893,"ongoing_decode_tokens":103893,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1914,"cache_hit":0,"new_prefill":14189,"score_linear":103893.0,"score_lmetric":14189},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2304,"cache_hit":11776,"new_prefill":2413,"score_linear":-11776.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2212,"cache_hit":0,"new_prefill":14189,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":0,"new_prefill":14189,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1749,"cache_hit":0,"new_prefill":14189,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":89469,"ongoing_decode_tokens":89469,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1704,"cache_hit":0,"new_prefill":14189,"score_linear":89469.0,"score_lmetric":14189},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":131699,"ongoing_decode_tokens":131699,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1707,"cache_hit":6144,"new_prefill":8045,"score_linear":125555.0,"score_lmetric":8045}],"chosen_score_linear":-11776.0,"chosen_score_lmetric":0,"t_first_token":349231.795441395,"t_first_token_unix":1779780096.809385,"t_done":349232.084715398,"t_done_unix":1779780097.0986583},{"request_id":"1313181:13:1349104:484","session_id":"1313181","input_length":131699,"t_proxy_recv":349190.374257035,"t_decision_unix":1779780055.3881984,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":65024,"affinity_cache_ratio":0.49373191899710706,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":65024,"estimated_new_tokens":66675,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1700,"cache_hit":24064,"new_prefill":107635,"score_linear":-24064.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1896,"cache_hit":0,"new_prefill":131699,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2304,"cache_hit":0,"new_prefill":131699,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2208,"cache_hit":0,"new_prefill":131699,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":0,"new_prefill":131699,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1749,"cache_hit":0,"new_prefill":131699,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1680,"cache_hit":65024,"new_prefill":66675,"score_linear":-65024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1707,"cache_hit":65024,"new_prefill":66675,"score_linear":-65024.0,"score_lmetric":0}],"chosen_score_linear":-65024.0,"chosen_score_lmetric":0,"t_first_token":349230.516973875,"t_first_token_unix":1779780095.5309176,"t_done":349232.201294167,"t_done_unix":1779780097.2152374},{"request_id":"1397515:2:1502033:923","session_id":"1397515","input_length":103893,"t_proxy_recv":349223.428344025,"t_decision_unix":1779780088.442285,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":93184,"affinity_cache_ratio":0.8969227955685176,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":93184,"estimated_new_tokens":10709,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1708,"cache_hit":0,"new_prefill":103893,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":75760,"ongoing_decode_tokens":75760,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1906,"cache_hit":93184,"new_prefill":10709,"score_linear":-17424.0,"score_lmetric":10709},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2304,"cache_hit":0,"new_prefill":103893,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2208,"cache_hit":0,"new_prefill":103893,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":0,"new_prefill":103893,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1749,"cache_hit":0,"new_prefill":103893,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":86984,"ongoing_decode_tokens":86984,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1698,"cache_hit":0,"new_prefill":103893,"score_linear":86984.0,"score_lmetric":103893},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":131699,"ongoing_decode_tokens":0,"pending_prefill_tokens":66675,"num_requests":1,"active_p_offloads":0,"cached_blocks":1707,"cache_hit":0,"new_prefill":103893,"score_linear":131699.0,"score_lmetric":170568}],"chosen_score_linear":-17424.0,"chosen_score_lmetric":10709,"t_first_token":349229.943687079,"t_first_token_unix":1779780094.9576302,"t_done":349233.972801354,"t_done_unix":1779780098.9867446},{"request_id":"1313181:13:1406095:783","session_id":"1313181","input_length":65279,"t_proxy_recv":349232.212519711,"t_decision_unix":1779780097.2264614,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":65024,"affinity_cache_ratio":0.9960936901606948,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":65024,"estimated_new_tokens":255,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1708,"cache_hit":24064,"new_prefill":41215,"score_linear":-24064.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":103893,"ongoing_decode_tokens":103893,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1914,"cache_hit":0,"new_prefill":65279,"score_linear":103893.0,"score_lmetric":65279},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2308,"cache_hit":0,"new_prefill":65279,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2212,"cache_hit":0,"new_prefill":65279,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":0,"new_prefill":65279,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1749,"cache_hit":0,"new_prefill":65279,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1709,"cache_hit":65024,"new_prefill":255,"score_linear":-65024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":65024,"new_prefill":255,"score_linear":-65024.0,"score_lmetric":0}],"chosen_score_linear":-65024.0,"chosen_score_lmetric":0,"t_first_token":349232.303379784,"t_first_token_unix":1779780097.3173227,"t_done":349234.482673391,"t_done_unix":1779780099.496617},{"request_id":"1363943:30:1506171:927","session_id":"1363943","input_length":89713,"t_proxy_recv":349237.308379369,"t_decision_unix":1779780102.3223205,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":89600,"affinity_cache_ratio":0.9987404278086788,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":89600,"estimated_new_tokens":113,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1708,"cache_hit":8704,"new_prefill":81009,"score_linear":-8704.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1935,"cache_hit":0,"new_prefill":89713,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2308,"cache_hit":0,"new_prefill":89713,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2212,"cache_hit":0,"new_prefill":89713,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":4096,"new_prefill":85617,"score_linear":-4096.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1749,"cache_hit":0,"new_prefill":89713,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1709,"cache_hit":89600,"new_prefill":113,"score_linear":-89600.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":139357,"ongoing_decode_tokens":0,"pending_prefill_tokens":7773,"num_requests":1,"active_p_offloads":0,"cached_blocks":1712,"cache_hit":4096,"new_prefill":85617,"score_linear":135261.0,"score_lmetric":93390}],"chosen_score_linear":-89600.0,"chosen_score_lmetric":0,"t_first_token":349237.451470281,"t_first_token_unix":1779780102.4654138,"t_done":349238.116140911,"t_done_unix":1779780103.1300843},{"request_id":"1268861:15:1508874:928","session_id":"1268861","input_length":63035,"t_proxy_recv":349246.366798832,"t_decision_unix":1779780111.3807402,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":61952,"affinity_cache_ratio":0.9828190687713175,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":61952,"estimated_new_tokens":1083,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1708,"cache_hit":0,"new_prefill":63035,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1935,"cache_hit":61952,"new_prefill":1083,"score_linear":-61952.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2308,"cache_hit":0,"new_prefill":63035,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2212,"cache_hit":0,"new_prefill":63035,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":0,"new_prefill":63035,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1749,"cache_hit":0,"new_prefill":63035,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1709,"cache_hit":0,"new_prefill":63035,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":139357,"ongoing_decode_tokens":0,"pending_prefill_tokens":7773,"num_requests":1,"active_p_offloads":0,"cached_blocks":1712,"cache_hit":12288,"new_prefill":50747,"score_linear":127069.0,"score_lmetric":58520}],"chosen_score_linear":-61952.0,"chosen_score_lmetric":0,"t_first_token":349246.94542262,"t_first_token_unix":1779780111.959366,"t_done":349248.389920155,"t_done_unix":1779780113.4038637},{"request_id":"1399948:15:1512300:930","session_id":"1399948","input_length":15669,"t_proxy_recv":349257.820886511,"t_decision_unix":1779780122.8348274,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":13824,"affinity_cache_ratio":0.8822515795519816,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":13824,"estimated_new_tokens":1845,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1708,"cache_hit":0,"new_prefill":15669,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1937,"cache_hit":0,"new_prefill":15669,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2308,"cache_hit":13824,"new_prefill":1845,"score_linear":-13824.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2212,"cache_hit":0,"new_prefill":15669,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":0,"new_prefill":15669,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1749,"cache_hit":0,"new_prefill":15669,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":89790,"ongoing_decode_tokens":89790,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1709,"cache_hit":0,"new_prefill":15669,"score_linear":89790.0,"score_lmetric":15669},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":139357,"ongoing_decode_tokens":0,"pending_prefill_tokens":7773,"num_requests":1,"active_p_offloads":0,"cached_blocks":1712,"cache_hit":6144,"new_prefill":9525,"score_linear":133213.0,"score_lmetric":17298}],"chosen_score_linear":-13824.0,"chosen_score_lmetric":0,"t_first_token":349258.060650965,"t_first_token_unix":1779780123.0745945,"t_done":349258.312932664,"t_done_unix":1779780123.326876},{"request_id":"1277428:9:1513231:932","session_id":"1277428","input_length":44364,"t_proxy_recv":349260.836316424,"t_decision_unix":1779780125.8502576,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":42496,"affinity_cache_ratio":0.9578937877558381,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":42496,"estimated_new_tokens":1868,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":130532,"ongoing_decode_tokens":130532,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1708,"cache_hit":0,"new_prefill":44364,"score_linear":130532.0,"score_lmetric":44364},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1937,"cache_hit":4608,"new_prefill":39756,"score_linear":-4608.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2311,"cache_hit":0,"new_prefill":44364,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2212,"cache_hit":17408,"new_prefill":26956,"score_linear":-17408.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":0,"new_prefill":44364,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1749,"cache_hit":42496,"new_prefill":1868,"score_linear":-42496.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":89790,"ongoing_decode_tokens":89790,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1709,"cache_hit":0,"new_prefill":44364,"score_linear":89790.0,"score_lmetric":44364},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":139357,"ongoing_decode_tokens":0,"pending_prefill_tokens":7773,"num_requests":1,"active_p_offloads":0,"cached_blocks":1712,"cache_hit":0,"new_prefill":44364,"score_linear":139357.0,"score_lmetric":52137}],"chosen_score_linear":-42496.0,"chosen_score_lmetric":0,"t_first_token":349261.35750547,"t_first_token_unix":1779780126.371449,"t_done":349261.660974223,"t_done_unix":1779780126.6749175},{"request_id":"1399948:16:1513839:933","session_id":"1399948","input_length":15913,"t_proxy_recv":349263.000637874,"t_decision_unix":1779780128.0145793,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":15360,"affinity_cache_ratio":0.9652485389304343,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":15360,"estimated_new_tokens":553,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":130532,"ongoing_decode_tokens":130532,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1708,"cache_hit":0,"new_prefill":15913,"score_linear":130532.0,"score_lmetric":15913},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1937,"cache_hit":0,"new_prefill":15913,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2311,"cache_hit":15360,"new_prefill":553,"score_linear":-15360.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2212,"cache_hit":0,"new_prefill":15913,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":0,"new_prefill":15913,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1752,"cache_hit":0,"new_prefill":15913,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":89790,"ongoing_decode_tokens":89790,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1709,"cache_hit":0,"new_prefill":15913,"score_linear":89790.0,"score_lmetric":15913},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":139357,"ongoing_decode_tokens":0,"pending_prefill_tokens":7773,"num_requests":1,"active_p_offloads":0,"cached_blocks":1712,"cache_hit":6144,"new_prefill":9769,"score_linear":133213.0,"score_lmetric":17542}],"chosen_score_linear":-15360.0,"chosen_score_lmetric":0,"t_first_token":349263.076754767,"t_first_token_unix":1779780128.0906985,"t_done":349263.512355534,"t_done_unix":1779780128.5262992},{"request_id":"1356885:4:1512902:931","session_id":"1356885","input_length":130532,"t_proxy_recv":349259.711836777,"t_decision_unix":1779780124.7257776,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":129536,"affinity_cache_ratio":0.9923696871265284,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":129536,"estimated_new_tokens":996,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1708,"cache_hit":129536,"new_prefill":996,"score_linear":-129536.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1937,"cache_hit":1024,"new_prefill":129508,"score_linear":-1024.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2311,"cache_hit":0,"new_prefill":130532,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2212,"cache_hit":0,"new_prefill":130532,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":6656,"new_prefill":123876,"score_linear":-6656.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1749,"cache_hit":0,"new_prefill":130532,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":89790,"ongoing_decode_tokens":89790,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1709,"cache_hit":0,"new_prefill":130532,"score_linear":89790.0,"score_lmetric":130532},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":139357,"ongoing_decode_tokens":0,"pending_prefill_tokens":7773,"num_requests":1,"active_p_offloads":0,"cached_blocks":1712,"cache_hit":0,"new_prefill":130532,"score_linear":139357.0,"score_lmetric":138305}],"chosen_score_linear":-129536.0,"chosen_score_lmetric":0,"t_first_token":349260.352476897,"t_first_token_unix":1779780125.3664207,"t_done":349263.794729946,"t_done_unix":1779780128.8086739},{"request_id":"1277428:10:1517408:935","session_id":"1277428","input_length":45851,"t_proxy_recv":349274.631548123,"t_decision_unix":1779780139.6454895,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":44032,"affinity_cache_ratio":0.9603280190181239,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":44032,"estimated_new_tokens":1819,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1710,"cache_hit":0,"new_prefill":45851,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1937,"cache_hit":4608,"new_prefill":41243,"score_linear":-4608.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2312,"cache_hit":0,"new_prefill":45851,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2212,"cache_hit":17408,"new_prefill":28443,"score_linear":-17408.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":0,"new_prefill":45851,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1752,"cache_hit":44032,"new_prefill":1819,"score_linear":-44032.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":89790,"ongoing_decode_tokens":89790,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1709,"cache_hit":0,"new_prefill":45851,"score_linear":89790.0,"score_lmetric":45851},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":139357,"ongoing_decode_tokens":0,"pending_prefill_tokens":7773,"num_requests":1,"active_p_offloads":0,"cached_blocks":1712,"cache_hit":0,"new_prefill":45851,"score_linear":139357.0,"score_lmetric":53624}],"chosen_score_linear":-44032.0,"chosen_score_lmetric":0,"t_first_token":349275.158226274,"t_first_token_unix":1779780140.1721706,"t_done":349277.262738198,"t_done_unix":1779780142.2766812},{"request_id":"1399948:17:1518102:936","session_id":"1399948","input_length":16604,"t_proxy_recv":349277.114465131,"t_decision_unix":1779780142.1284068,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":15872,"affinity_cache_ratio":0.9559142375331245,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":15872,"estimated_new_tokens":732,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1710,"cache_hit":0,"new_prefill":16604,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1937,"cache_hit":0,"new_prefill":16604,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2312,"cache_hit":15872,"new_prefill":732,"score_linear":-15872.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2212,"cache_hit":0,"new_prefill":16604,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":0,"new_prefill":16604,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":45851,"ongoing_decode_tokens":45851,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1752,"cache_hit":0,"new_prefill":16604,"score_linear":45851.0,"score_lmetric":16604},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":89790,"ongoing_decode_tokens":89790,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1709,"cache_hit":0,"new_prefill":16604,"score_linear":89790.0,"score_lmetric":16604},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":139357,"ongoing_decode_tokens":0,"pending_prefill_tokens":7773,"num_requests":1,"active_p_offloads":0,"cached_blocks":1712,"cache_hit":6144,"new_prefill":10460,"score_linear":133213.0,"score_lmetric":18233}],"chosen_score_linear":-15872.0,"chosen_score_lmetric":0,"t_first_token":349277.248777822,"t_first_token_unix":1779780142.2627208,"t_done":349277.607267144,"t_done_unix":1779780142.6212106},{"request_id":"1363943:31:1509815:929","session_id":"1363943","input_length":89790,"t_proxy_recv":349249.385300508,"t_decision_unix":1779780114.399242,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":89600,"affinity_cache_ratio":0.9978839514422542,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":89600,"estimated_new_tokens":190,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1708,"cache_hit":8704,"new_prefill":81086,"score_linear":-8704.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1937,"cache_hit":0,"new_prefill":89790,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2308,"cache_hit":0,"new_prefill":89790,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2212,"cache_hit":0,"new_prefill":89790,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":4096,"new_prefill":85694,"score_linear":-4096.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1749,"cache_hit":0,"new_prefill":89790,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1709,"cache_hit":89600,"new_prefill":190,"score_linear":-89600.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":139357,"ongoing_decode_tokens":0,"pending_prefill_tokens":7773,"num_requests":1,"active_p_offloads":0,"cached_blocks":1712,"cache_hit":4096,"new_prefill":85694,"score_linear":135261.0,"score_lmetric":93467}],"chosen_score_linear":-89600.0,"chosen_score_lmetric":0,"t_first_token":349249.506989246,"t_first_token_unix":1779780114.5209327,"t_done":349284.930361758,"t_done_unix":1779780149.9443052},{"request_id":"1277428:11:1520449:937","session_id":"1277428","input_length":46361,"t_proxy_recv":349284.624071212,"t_decision_unix":1779780149.6380124,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":46080,"affinity_cache_ratio":0.9939388710338432,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":46080,"estimated_new_tokens":281,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1710,"cache_hit":0,"new_prefill":46361,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1937,"cache_hit":4608,"new_prefill":41753,"score_linear":-4608.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2313,"cache_hit":0,"new_prefill":46361,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2212,"cache_hit":17408,"new_prefill":28953,"score_linear":-17408.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":0,"new_prefill":46361,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1756,"cache_hit":46080,"new_prefill":281,"score_linear":-46080.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":89790,"ongoing_decode_tokens":89790,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1709,"cache_hit":0,"new_prefill":46361,"score_linear":89790.0,"score_lmetric":46361},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":139357,"ongoing_decode_tokens":139357,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1712,"cache_hit":0,"new_prefill":46361,"score_linear":139357.0,"score_lmetric":46361}],"chosen_score_linear":-46080.0,"chosen_score_lmetric":0,"t_first_token":349284.789260684,"t_first_token_unix":1779780149.803204,"t_done":349285.189110533,"t_done_unix":1779780150.2030542},{"request_id":"1313181:14:1351342:491","session_id":"1313181","input_length":139357,"t_proxy_recv":349234.512244524,"t_decision_unix":1779780099.526186,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":131584,"affinity_cache_ratio":0.9442223928471479,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":131584,"estimated_new_tokens":7773,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1708,"cache_hit":24064,"new_prefill":115293,"score_linear":-24064.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1935,"cache_hit":0,"new_prefill":139357,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2308,"cache_hit":0,"new_prefill":139357,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2212,"cache_hit":0,"new_prefill":139357,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":0,"new_prefill":139357,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1749,"cache_hit":0,"new_prefill":139357,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1709,"cache_hit":65024,"new_prefill":74333,"score_linear":-65024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1712,"cache_hit":131584,"new_prefill":7773,"score_linear":-131584.0,"score_lmetric":0}],"chosen_score_linear":-131584.0,"chosen_score_lmetric":0,"t_first_token":349280.59128428,"t_first_token_unix":1779780145.6052341,"t_done":349288.946080122,"t_done_unix":1779780153.960024},{"request_id":"1313181:14:1408792:803","session_id":"1313181","input_length":65876,"t_proxy_recv":349288.95656387,"t_decision_unix":1779780153.9705052,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":65536,"affinity_cache_ratio":0.9948387880259882,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":65536,"estimated_new_tokens":340,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1710,"cache_hit":24064,"new_prefill":41812,"score_linear":-24064.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1937,"cache_hit":0,"new_prefill":65876,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2313,"cache_hit":0,"new_prefill":65876,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2212,"cache_hit":0,"new_prefill":65876,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":0,"new_prefill":65876,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1756,"cache_hit":0,"new_prefill":65876,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1710,"cache_hit":65024,"new_prefill":852,"score_linear":-65024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1728,"cache_hit":65536,"new_prefill":340,"score_linear":-65536.0,"score_lmetric":0}],"chosen_score_linear":-65536.0,"chosen_score_lmetric":0,"t_first_token":349289.046261815,"t_first_token_unix":1779780154.0602052,"t_done":349292.028919497,"t_done_unix":1779780157.0428634},{"request_id":"1313181:15:1411723:818","session_id":"1313181","input_length":66259,"t_proxy_recv":349292.04057557,"t_decision_unix":1779780157.054517,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":7,"chosen_idx":7,"affinity_cache_hit":66048,"affinity_cache_ratio":0.9968155269472826,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":66048,"estimated_new_tokens":211,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8007","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1710,"cache_hit":24064,"new_prefill":42195,"score_linear":-24064.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1937,"cache_hit":0,"new_prefill":66259,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2313,"cache_hit":0,"new_prefill":66259,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2212,"cache_hit":0,"new_prefill":66259,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":0,"new_prefill":66259,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1756,"cache_hit":0,"new_prefill":66259,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1710,"cache_hit":65024,"new_prefill":1235,"score_linear":-65024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1729,"cache_hit":66048,"new_prefill":211,"score_linear":-66048.0,"score_lmetric":0}],"chosen_score_linear":-66048.0,"chosen_score_lmetric":0,"t_first_token":349292.148082892,"t_first_token_unix":1779780157.1620262,"t_done":349295.406325765,"t_done_unix":1779780160.4202695},{"request_id":"1399948:18:1524432:938","session_id":"1399948","input_length":17276,"t_proxy_recv":349298.167186312,"t_decision_unix":1779780163.1811273,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":16384,"affinity_cache_ratio":0.948367677703172,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":16384,"estimated_new_tokens":892,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1710,"cache_hit":0,"new_prefill":17276,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1937,"cache_hit":0,"new_prefill":17276,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2313,"cache_hit":16384,"new_prefill":892,"score_linear":-16384.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2212,"cache_hit":0,"new_prefill":17276,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":0,"new_prefill":17276,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1756,"cache_hit":0,"new_prefill":17276,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1710,"cache_hit":0,"new_prefill":17276,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":6144,"new_prefill":11132,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-16384.0,"chosen_score_lmetric":0,"t_first_token":349298.299229386,"t_first_token_unix":1779780163.3131728,"t_done":349298.657162453,"t_done_unix":1779780163.6711056},{"request_id":"1342921:9:1525898:939","session_id":"1342921","input_length":79086,"t_proxy_recv":349303.209055275,"t_decision_unix":1779780168.2229965,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":78336,"affinity_cache_ratio":0.9905166527577574,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":78336,"estimated_new_tokens":750,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1710,"cache_hit":0,"new_prefill":79086,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1937,"cache_hit":0,"new_prefill":79086,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2314,"cache_hit":0,"new_prefill":79086,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2212,"cache_hit":78336,"new_prefill":750,"score_linear":-78336.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":0,"new_prefill":79086,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1756,"cache_hit":0,"new_prefill":79086,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1710,"cache_hit":0,"new_prefill":79086,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":79086,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-78336.0,"chosen_score_lmetric":0,"t_first_token":349303.636704176,"t_first_token_unix":1779780168.6506479,"t_done":349303.696237591,"t_done_unix":1779780168.7101805},{"request_id":"1399948:19:1526739:941","session_id":"1399948","input_length":18091,"t_proxy_recv":349306.09206629,"t_decision_unix":1779780171.1060076,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":16896,"affinity_cache_ratio":0.9339450555524846,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":16896,"estimated_new_tokens":1195,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1710,"cache_hit":0,"new_prefill":18091,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":87576,"ongoing_decode_tokens":0,"pending_prefill_tokens":10776,"num_requests":1,"active_p_offloads":0,"cached_blocks":1937,"cache_hit":0,"new_prefill":18091,"score_linear":87576.0,"score_lmetric":28867},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2314,"cache_hit":16896,"new_prefill":1195,"score_linear":-16896.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2213,"cache_hit":0,"new_prefill":18091,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":0,"new_prefill":18091,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1756,"cache_hit":0,"new_prefill":18091,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1710,"cache_hit":0,"new_prefill":18091,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":6144,"new_prefill":11947,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-16896.0,"chosen_score_lmetric":0,"t_first_token":349306.266298648,"t_first_token_unix":1779780171.280242,"t_done":349306.552161272,"t_done_unix":1779780171.5661047},{"request_id":"1399948:20:1527857:942","session_id":"1399948","input_length":18164,"t_proxy_recv":349309.585471487,"t_decision_unix":1779780174.599413,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":17920,"affinity_cache_ratio":0.9865668354987888,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":17920,"estimated_new_tokens":244,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1710,"cache_hit":0,"new_prefill":18164,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":87576,"ongoing_decode_tokens":0,"pending_prefill_tokens":10776,"num_requests":1,"active_p_offloads":0,"cached_blocks":1937,"cache_hit":0,"new_prefill":18164,"score_linear":87576.0,"score_lmetric":28940},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2316,"cache_hit":17920,"new_prefill":244,"score_linear":-17920.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2213,"cache_hit":0,"new_prefill":18164,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":0,"new_prefill":18164,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1756,"cache_hit":0,"new_prefill":18164,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1710,"cache_hit":0,"new_prefill":18164,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":6144,"new_prefill":12020,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-17920.0,"chosen_score_lmetric":0,"t_first_token":349309.634321233,"t_first_token_unix":1779780174.6482644,"t_done":349310.176371643,"t_done_unix":1779780175.1903148},{"request_id":"1399948:21:1528978:944","session_id":"1399948","input_length":18388,"t_proxy_recv":349313.531956649,"t_decision_unix":1779780178.5458977,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":17920,"affinity_cache_ratio":0.9745486186643463,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":17920,"estimated_new_tokens":468,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":36568,"ongoing_decode_tokens":0,"pending_prefill_tokens":36568,"num_requests":1,"active_p_offloads":0,"cached_blocks":1710,"cache_hit":0,"new_prefill":18388,"score_linear":36568.0,"score_lmetric":54956},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":87576,"ongoing_decode_tokens":0,"pending_prefill_tokens":10776,"num_requests":1,"active_p_offloads":0,"cached_blocks":1937,"cache_hit":0,"new_prefill":18388,"score_linear":87576.0,"score_lmetric":29164},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2316,"cache_hit":17920,"new_prefill":468,"score_linear":-17920.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2213,"cache_hit":0,"new_prefill":18388,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":0,"new_prefill":18388,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1756,"cache_hit":0,"new_prefill":18388,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1710,"cache_hit":0,"new_prefill":18388,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":6144,"new_prefill":12244,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-17920.0,"chosen_score_lmetric":0,"t_first_token":349313.612741966,"t_first_token_unix":1779780178.6266851,"t_done":349314.202865902,"t_done_unix":1779780179.2168093},{"request_id":"1399948:22:1530122:947","session_id":"1399948","input_length":18563,"t_proxy_recv":349317.360860598,"t_decision_unix":1779780182.3748019,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":18432,"affinity_cache_ratio":0.992942951031622,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":18432,"estimated_new_tokens":131,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":36568,"ongoing_decode_tokens":36568,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1710,"cache_hit":0,"new_prefill":18563,"score_linear":36568.0,"score_lmetric":18563},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":158768,"ongoing_decode_tokens":0,"pending_prefill_tokens":18992,"num_requests":2,"active_p_offloads":0,"cached_blocks":1937,"cache_hit":0,"new_prefill":18563,"score_linear":158768.0,"score_lmetric":75110},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2317,"cache_hit":18432,"new_prefill":131,"score_linear":-18432.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2213,"cache_hit":0,"new_prefill":18563,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":0,"new_prefill":18563,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1756,"cache_hit":0,"new_prefill":18563,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1710,"cache_hit":0,"new_prefill":18563,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":6144,"new_prefill":12419,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-18432.0,"chosen_score_lmetric":0,"t_first_token":349317.421938086,"t_first_token_unix":1779780182.4358814,"t_done":349317.672308367,"t_done_unix":1779780182.6862514},{"request_id":"1243831:2:1528168:943","session_id":"1243831","input_length":36568,"t_proxy_recv":349310.792344175,"t_decision_unix":1779780175.8062856,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":1,"chosen_idx":0,"affinity_cache_hit":11264,"affinity_cache_ratio":0.30802887770728504,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":true,"cache_hit":0,"estimated_new_tokens":36568,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1710,"cache_hit":0,"new_prefill":36568,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":87576,"ongoing_decode_tokens":0,"pending_prefill_tokens":10776,"num_requests":1,"active_p_offloads":0,"cached_blocks":1937,"cache_hit":11264,"new_prefill":25304,"score_linear":76312.0,"score_lmetric":36080},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2316,"cache_hit":0,"new_prefill":36568,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2213,"cache_hit":0,"new_prefill":36568,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":0,"new_prefill":36568,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1756,"cache_hit":0,"new_prefill":36568,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1710,"cache_hit":0,"new_prefill":36568,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":36568,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":0.0,"chosen_score_lmetric":0,"t_first_token":349316.266491152,"t_first_token_unix":1779780181.280444,"t_done":349320.562635659,"t_done_unix":1779780185.576579},{"request_id":"1243831:3:1270336:110","session_id":"1243831","input_length":11584,"t_proxy_recv":349320.566136081,"t_decision_unix":1779780185.580078,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":11264,"affinity_cache_ratio":0.9723756906077348,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":11264,"estimated_new_tokens":320,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1782,"cache_hit":11264,"new_prefill":320,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":158768,"ongoing_decode_tokens":0,"pending_prefill_tokens":18992,"num_requests":2,"active_p_offloads":0,"cached_blocks":1937,"cache_hit":11264,"new_prefill":320,"score_linear":147504.0,"score_lmetric":38624},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2317,"cache_hit":0,"new_prefill":11584,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2213,"cache_hit":0,"new_prefill":11584,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":0,"new_prefill":11584,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1756,"cache_hit":0,"new_prefill":11584,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1710,"cache_hit":0,"new_prefill":11584,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":11584,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-11264.0,"chosen_score_lmetric":0,"t_first_token":349320.59591897,"t_first_token_unix":1779780185.6098619,"t_done":349321.441802062,"t_done_unix":1779780186.4557452},{"request_id":"1243831:4:1284306:183","session_id":"1243831","input_length":14515,"t_proxy_recv":349321.447984082,"t_decision_unix":1779780186.4619257,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":11264,"affinity_cache_ratio":0.7760248019290389,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":11264,"estimated_new_tokens":3251,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1782,"cache_hit":11264,"new_prefill":3251,"score_linear":-11264.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":158768,"ongoing_decode_tokens":0,"pending_prefill_tokens":18992,"num_requests":2,"active_p_offloads":0,"cached_blocks":1937,"cache_hit":11264,"new_prefill":3251,"score_linear":147504.0,"score_lmetric":44486},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2317,"cache_hit":0,"new_prefill":14515,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2213,"cache_hit":0,"new_prefill":14515,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":0,"new_prefill":14515,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1756,"cache_hit":0,"new_prefill":14515,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1710,"cache_hit":0,"new_prefill":14515,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":14515,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-11264.0,"chosen_score_lmetric":0,"t_first_token":349321.815032306,"t_first_token_unix":1779780186.8289754,"t_done":349322.247900078,"t_done_unix":1779780187.261843},{"request_id":"1243831:5:1303288:273","session_id":"1243831","input_length":14627,"t_proxy_recv":349322.252319629,"t_decision_unix":1779780187.2662616,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":14336,"affinity_cache_ratio":0.980105284747385,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":14336,"estimated_new_tokens":291,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":14336,"new_prefill":291,"score_linear":-14336.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":158768,"ongoing_decode_tokens":0,"pending_prefill_tokens":18992,"num_requests":2,"active_p_offloads":0,"cached_blocks":1937,"cache_hit":11264,"new_prefill":3363,"score_linear":147504.0,"score_lmetric":44710},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2317,"cache_hit":0,"new_prefill":14627,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2213,"cache_hit":0,"new_prefill":14627,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":0,"new_prefill":14627,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1756,"cache_hit":0,"new_prefill":14627,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1710,"cache_hit":0,"new_prefill":14627,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":14627,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-14336.0,"chosen_score_lmetric":0,"t_first_token":349322.297325772,"t_first_token_unix":1779780187.3112688,"t_done":349322.753301449,"t_done_unix":1779780187.7672446},{"request_id":"1243831:6:1313694:320","session_id":"1243831","input_length":17410,"t_proxy_recv":349322.759445147,"t_decision_unix":1779780187.773387,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":14336,"affinity_cache_ratio":0.8234348075818495,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":14336,"estimated_new_tokens":3074,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1788,"cache_hit":14336,"new_prefill":3074,"score_linear":-14336.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":158768,"ongoing_decode_tokens":0,"pending_prefill_tokens":18992,"num_requests":2,"active_p_offloads":0,"cached_blocks":1937,"cache_hit":11264,"new_prefill":6146,"score_linear":147504.0,"score_lmetric":50276},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2317,"cache_hit":0,"new_prefill":17410,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2213,"cache_hit":0,"new_prefill":17410,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":0,"new_prefill":17410,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1756,"cache_hit":0,"new_prefill":17410,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1710,"cache_hit":0,"new_prefill":17410,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":17410,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-14336.0,"chosen_score_lmetric":0,"t_first_token":349323.169971183,"t_first_token_unix":1779780188.1839144,"t_done":349323.629431643,"t_done_unix":1779780188.6433747},{"request_id":"1243831:7:1323865:366","session_id":"1243831","input_length":19460,"t_proxy_recv":349323.634811934,"t_decision_unix":1779780188.6487536,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":17408,"affinity_cache_ratio":0.8945529290853032,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":17408,"estimated_new_tokens":2052,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1794,"cache_hit":17408,"new_prefill":2052,"score_linear":-17408.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":158768,"ongoing_decode_tokens":0,"pending_prefill_tokens":18992,"num_requests":2,"active_p_offloads":0,"cached_blocks":1937,"cache_hit":11264,"new_prefill":8196,"score_linear":147504.0,"score_lmetric":54376},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2317,"cache_hit":0,"new_prefill":19460,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2213,"cache_hit":0,"new_prefill":19460,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":0,"new_prefill":19460,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1756,"cache_hit":0,"new_prefill":19460,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1710,"cache_hit":0,"new_prefill":19460,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":19460,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-17408.0,"chosen_score_lmetric":0,"t_first_token":349323.991789805,"t_first_token_unix":1779780189.005733,"t_done":349324.594283691,"t_done_unix":1779780189.6082268},{"request_id":"1243831:8:1333049:414","session_id":"1243831","input_length":24512,"t_proxy_recv":349324.60187647,"t_decision_unix":1779780189.615818,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":19456,"affinity_cache_ratio":0.793733681462141,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":19456,"estimated_new_tokens":5056,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1798,"cache_hit":19456,"new_prefill":5056,"score_linear":-19456.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":158768,"ongoing_decode_tokens":0,"pending_prefill_tokens":18992,"num_requests":2,"active_p_offloads":0,"cached_blocks":1937,"cache_hit":11264,"new_prefill":13248,"score_linear":147504.0,"score_lmetric":64480},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2317,"cache_hit":0,"new_prefill":24512,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2213,"cache_hit":0,"new_prefill":24512,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":0,"new_prefill":24512,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1756,"cache_hit":0,"new_prefill":24512,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1710,"cache_hit":0,"new_prefill":24512,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":24512,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-19456.0,"chosen_score_lmetric":0,"t_first_token":349325.46925349,"t_first_token_unix":1779780190.483197,"t_done":349326.151824671,"t_done_unix":1779780191.165768},{"request_id":"1243831:9:1348240:478","session_id":"1243831","input_length":25075,"t_proxy_recv":349326.157841565,"t_decision_unix":1779780191.1717832,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":24576,"affinity_cache_ratio":0.980099700897308,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":24576,"estimated_new_tokens":499,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1808,"cache_hit":24576,"new_prefill":499,"score_linear":-24576.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":158768,"ongoing_decode_tokens":0,"pending_prefill_tokens":18992,"num_requests":2,"active_p_offloads":0,"cached_blocks":1937,"cache_hit":11264,"new_prefill":13811,"score_linear":147504.0,"score_lmetric":65606},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2317,"cache_hit":0,"new_prefill":25075,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2213,"cache_hit":0,"new_prefill":25075,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":0,"new_prefill":25075,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1756,"cache_hit":0,"new_prefill":25075,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1710,"cache_hit":0,"new_prefill":25075,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":25075,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-24576.0,"chosen_score_lmetric":0,"t_first_token":349326.31620281,"t_first_token_unix":1779780191.3301458,"t_done":349326.727744155,"t_done_unix":1779780191.741687},{"request_id":"1243831:10:1358921:521","session_id":"1243831","input_length":26343,"t_proxy_recv":349326.734111693,"t_decision_unix":1779780191.7480533,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":25088,"affinity_cache_ratio":0.9523592605246175,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":25088,"estimated_new_tokens":1255,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1809,"cache_hit":25088,"new_prefill":1255,"score_linear":-25088.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":158768,"ongoing_decode_tokens":0,"pending_prefill_tokens":18992,"num_requests":2,"active_p_offloads":0,"cached_blocks":1937,"cache_hit":11264,"new_prefill":15079,"score_linear":147504.0,"score_lmetric":68142},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2317,"cache_hit":0,"new_prefill":26343,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2213,"cache_hit":0,"new_prefill":26343,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":0,"new_prefill":26343,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1756,"cache_hit":0,"new_prefill":26343,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1710,"cache_hit":0,"new_prefill":26343,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":26343,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-25088.0,"chosen_score_lmetric":0,"t_first_token":349327.01825337,"t_first_token_unix":1779780192.0321968,"t_done":349327.721260064,"t_done_unix":1779780192.735203},{"request_id":"1243831:11:1371191:594","session_id":"1243831","input_length":26818,"t_proxy_recv":349327.728267841,"t_decision_unix":1779780192.7422092,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":26112,"affinity_cache_ratio":0.9736743977925274,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":26112,"estimated_new_tokens":706,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1811,"cache_hit":26112,"new_prefill":706,"score_linear":-26112.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":158768,"ongoing_decode_tokens":0,"pending_prefill_tokens":18992,"num_requests":2,"active_p_offloads":0,"cached_blocks":1937,"cache_hit":11264,"new_prefill":15554,"score_linear":147504.0,"score_lmetric":69092},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2317,"cache_hit":0,"new_prefill":26818,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2213,"cache_hit":0,"new_prefill":26818,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":0,"new_prefill":26818,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1756,"cache_hit":0,"new_prefill":26818,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1710,"cache_hit":0,"new_prefill":26818,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":26818,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-26112.0,"chosen_score_lmetric":0,"t_first_token":349327.900579131,"t_first_token_unix":1779780192.914523,"t_done":349328.354602041,"t_done_unix":1779780193.3685596},{"request_id":"1243831:12:1385174:681","session_id":"1243831","input_length":27164,"t_proxy_recv":349328.361249653,"t_decision_unix":1779780193.375191,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":26624,"affinity_cache_ratio":0.9801207480488883,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":26624,"estimated_new_tokens":540,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1812,"cache_hit":26624,"new_prefill":540,"score_linear":-26624.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":158768,"ongoing_decode_tokens":0,"pending_prefill_tokens":18992,"num_requests":2,"active_p_offloads":0,"cached_blocks":1937,"cache_hit":11264,"new_prefill":15900,"score_linear":147504.0,"score_lmetric":69784},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2317,"cache_hit":0,"new_prefill":27164,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2213,"cache_hit":0,"new_prefill":27164,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":0,"new_prefill":27164,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1756,"cache_hit":0,"new_prefill":27164,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1710,"cache_hit":0,"new_prefill":27164,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":27164,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-26624.0,"chosen_score_lmetric":0,"t_first_token":349328.472402134,"t_first_token_unix":1779780193.4863453,"t_done":349328.884653875,"t_done_unix":1779780193.8985968},{"request_id":"1243831:13:1398072:738","session_id":"1243831","input_length":27286,"t_proxy_recv":349328.890571223,"t_decision_unix":1779780193.9045131,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":27136,"affinity_cache_ratio":0.9945026753646559,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":27136,"estimated_new_tokens":150,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1813,"cache_hit":27136,"new_prefill":150,"score_linear":-27136.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":158768,"ongoing_decode_tokens":0,"pending_prefill_tokens":18992,"num_requests":2,"active_p_offloads":0,"cached_blocks":1937,"cache_hit":11264,"new_prefill":16022,"score_linear":147504.0,"score_lmetric":70028},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2317,"cache_hit":0,"new_prefill":27286,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2213,"cache_hit":0,"new_prefill":27286,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":0,"new_prefill":27286,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1756,"cache_hit":0,"new_prefill":27286,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1710,"cache_hit":0,"new_prefill":27286,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":27286,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-27136.0,"chosen_score_lmetric":0,"t_first_token":349328.961279695,"t_first_token_unix":1779780193.975223,"t_done":349329.923432807,"t_done_unix":1779780194.9373765},{"request_id":"1243831:14:1411189:814","session_id":"1243831","input_length":28085,"t_proxy_recv":349329.929754572,"t_decision_unix":1779780194.943696,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":27136,"affinity_cache_ratio":0.9662097204913654,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":27136,"estimated_new_tokens":949,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":130906,"ongoing_decode_tokens":130906,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1813,"cache_hit":27136,"new_prefill":949,"score_linear":103770.0,"score_lmetric":949},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":158768,"ongoing_decode_tokens":0,"pending_prefill_tokens":18992,"num_requests":2,"active_p_offloads":0,"cached_blocks":1937,"cache_hit":11264,"new_prefill":16821,"score_linear":147504.0,"score_lmetric":71626},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2317,"cache_hit":0,"new_prefill":28085,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2213,"cache_hit":0,"new_prefill":28085,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":71987,"ongoing_decode_tokens":0,"pending_prefill_tokens":7987,"num_requests":1,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":0,"new_prefill":28085,"score_linear":71987.0,"score_lmetric":36072},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1756,"cache_hit":0,"new_prefill":28085,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1710,"cache_hit":0,"new_prefill":28085,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":28085,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":103770.0,"chosen_score_lmetric":949,"t_first_token":349330.249916961,"t_first_token_unix":1779780195.2638612,"t_done":349331.462831614,"t_done_unix":1779780196.4767754},{"request_id":"1356885:5:1533696:948","session_id":"1356885","input_length":130906,"t_proxy_recv":349329.202724214,"t_decision_unix":1779780194.2166655,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":130560,"affinity_cache_ratio":0.9973568820374925,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":130560,"estimated_new_tokens":346,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":27286,"ongoing_decode_tokens":27286,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1813,"cache_hit":130560,"new_prefill":346,"score_linear":-103274.0,"score_lmetric":346},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":158768,"ongoing_decode_tokens":0,"pending_prefill_tokens":18992,"num_requests":2,"active_p_offloads":0,"cached_blocks":1937,"cache_hit":1024,"new_prefill":129882,"score_linear":157744.0,"score_lmetric":297748},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2317,"cache_hit":0,"new_prefill":130906,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2213,"cache_hit":0,"new_prefill":130906,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":6656,"new_prefill":124250,"score_linear":-6656.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1756,"cache_hit":0,"new_prefill":130906,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1710,"cache_hit":0,"new_prefill":130906,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":130906,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-103274.0,"chosen_score_lmetric":346,"t_first_token":349329.452578368,"t_first_token_unix":1779780194.4665222,"t_done":349332.657233968,"t_done_unix":1779780197.671178},{"request_id":"1243831:15:1427593:849","session_id":"1243831","input_length":29382,"t_proxy_recv":349331.469556942,"t_decision_unix":1779780196.4834986,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":28160,"affinity_cache_ratio":0.9584099108297597,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":28160,"estimated_new_tokens":1222,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":130906,"ongoing_decode_tokens":130906,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1815,"cache_hit":28160,"new_prefill":1222,"score_linear":102746.0,"score_lmetric":1222},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":158768,"ongoing_decode_tokens":0,"pending_prefill_tokens":18992,"num_requests":2,"active_p_offloads":0,"cached_blocks":1937,"cache_hit":11264,"new_prefill":18118,"score_linear":147504.0,"score_lmetric":74220},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2317,"cache_hit":0,"new_prefill":29382,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2213,"cache_hit":0,"new_prefill":29382,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":71987,"ongoing_decode_tokens":0,"pending_prefill_tokens":7987,"num_requests":1,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":0,"new_prefill":29382,"score_linear":71987.0,"score_lmetric":37369},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1756,"cache_hit":0,"new_prefill":29382,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1710,"cache_hit":0,"new_prefill":29382,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":29382,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":102746.0,"chosen_score_lmetric":1222,"t_first_token":349331.834515711,"t_first_token_unix":1779780196.8484597,"t_done":349333.155474027,"t_done_unix":1779780198.1694171},{"request_id":"1243831:16:1446717:874","session_id":"1243831","input_length":29571,"t_proxy_recv":349333.161294622,"t_decision_unix":1779780198.175236,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":29184,"affinity_cache_ratio":0.9869128538094755,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":29184,"estimated_new_tokens":387,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1818,"cache_hit":29184,"new_prefill":387,"score_linear":-29184.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":158768,"ongoing_decode_tokens":87576,"pending_prefill_tokens":8216,"num_requests":2,"active_p_offloads":0,"cached_blocks":1937,"cache_hit":11264,"new_prefill":18307,"score_linear":147504.0,"score_lmetric":53046},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2317,"cache_hit":0,"new_prefill":29571,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2213,"cache_hit":0,"new_prefill":29571,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":71987,"ongoing_decode_tokens":71987,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":0,"new_prefill":29571,"score_linear":71987.0,"score_lmetric":29571},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1756,"cache_hit":0,"new_prefill":29571,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1710,"cache_hit":0,"new_prefill":29571,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":29571,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-29184.0,"chosen_score_lmetric":0,"t_first_token":349333.238942883,"t_first_token_unix":1779780198.252886,"t_done":349333.672285604,"t_done_unix":1779780198.686229},{"request_id":"1294611:15:1533841:949","session_id":"1294611","input_length":71987,"t_proxy_recv":349329.66838482,"t_decision_unix":1779780194.6823258,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":64000,"affinity_cache_ratio":0.8890494116993346,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":64000,"estimated_new_tokens":7987,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":158192,"ongoing_decode_tokens":158192,"pending_prefill_tokens":0,"num_requests":2,"active_p_offloads":0,"cached_blocks":1813,"cache_hit":0,"new_prefill":71987,"score_linear":158192.0,"score_lmetric":143974},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":158768,"ongoing_decode_tokens":0,"pending_prefill_tokens":18992,"num_requests":2,"active_p_offloads":0,"cached_blocks":1937,"cache_hit":0,"new_prefill":71987,"score_linear":158768.0,"score_lmetric":181958},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2317,"cache_hit":0,"new_prefill":71987,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2213,"cache_hit":0,"new_prefill":71987,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":64000,"new_prefill":7987,"score_linear":-64000.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1756,"cache_hit":12800,"new_prefill":59187,"score_linear":-12800.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1710,"cache_hit":0,"new_prefill":71987,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":12800,"new_prefill":59187,"score_linear":-12800.0,"score_lmetric":0}],"chosen_score_linear":-64000.0,"chosen_score_lmetric":0,"t_first_token":349333.128966171,"t_first_token_unix":1779780198.1429102,"t_done":349335.916987776,"t_done_unix":1779780200.930931},{"request_id":"1268861:16:1529894:946","session_id":"1268861","input_length":71192,"t_proxy_recv":349316.547200331,"t_decision_unix":1779780181.5611417,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":62976,"affinity_cache_ratio":0.8845937745814136,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":62976,"estimated_new_tokens":8216,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":36568,"ongoing_decode_tokens":36568,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1710,"cache_hit":0,"new_prefill":71192,"score_linear":36568.0,"score_lmetric":71192},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":87576,"ongoing_decode_tokens":0,"pending_prefill_tokens":10776,"num_requests":1,"active_p_offloads":0,"cached_blocks":1937,"cache_hit":62976,"new_prefill":8216,"score_linear":24600.0,"score_lmetric":18992},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2317,"cache_hit":0,"new_prefill":71192,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2213,"cache_hit":0,"new_prefill":71192,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":0,"new_prefill":71192,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1756,"cache_hit":0,"new_prefill":71192,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1710,"cache_hit":0,"new_prefill":71192,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":12288,"new_prefill":58904,"score_linear":-12288.0,"score_lmetric":0}],"chosen_score_linear":24600.0,"chosen_score_lmetric":18992,"t_first_token":349334.853625333,"t_first_token_unix":1779780199.8675706,"t_done":349336.708156237,"t_done_unix":1779780201.7220998},{"request_id":"1363943:32:1537767:950","session_id":"1363943","input_length":93368,"t_proxy_recv":349342.992441637,"t_decision_unix":1779780208.0063822,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":93184,"affinity_cache_ratio":0.9980293034015937,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":93184,"estimated_new_tokens":184,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":31161,"ongoing_decode_tokens":31161,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1818,"cache_hit":8704,"new_prefill":84664,"score_linear":22457.0,"score_lmetric":84664},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":87576,"ongoing_decode_tokens":87576,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1953,"cache_hit":0,"new_prefill":93368,"score_linear":87576.0,"score_lmetric":93368},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2317,"cache_hit":0,"new_prefill":93368,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2213,"cache_hit":0,"new_prefill":93368,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2751,"cache_hit":4096,"new_prefill":89272,"score_linear":-4096.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1756,"cache_hit":0,"new_prefill":93368,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1710,"cache_hit":93184,"new_prefill":184,"score_linear":-93184.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":4096,"new_prefill":89272,"score_linear":-4096.0,"score_lmetric":0}],"chosen_score_linear":-93184.0,"chosen_score_lmetric":0,"t_first_token":349343.1953363,"t_first_token_unix":1779780208.20928,"t_done":349345.132079609,"t_done_unix":1779780210.1460235},{"request_id":"1253804:7:1526643:940","session_id":"1253804","input_length":87576,"t_proxy_recv":349305.653463409,"t_decision_unix":1779780170.6674047,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":76800,"affinity_cache_ratio":0.8769525897506166,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":76800,"estimated_new_tokens":10776,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1710,"cache_hit":0,"new_prefill":87576,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1937,"cache_hit":76800,"new_prefill":10776,"score_linear":-76800.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2314,"cache_hit":0,"new_prefill":87576,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2213,"cache_hit":3584,"new_prefill":83992,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":0,"new_prefill":87576,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1756,"cache_hit":3584,"new_prefill":83992,"score_linear":-3584.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1710,"cache_hit":0,"new_prefill":87576,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":87576,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-76800.0,"chosen_score_lmetric":0,"t_first_token":349332.47172146,"t_first_token_unix":1779780197.485666,"t_done":349350.271975664,"t_done_unix":1779780215.285919},{"request_id":"1253804:8:1541192:953","session_id":"1253804","input_length":89116,"t_proxy_recv":349354.140879402,"t_decision_unix":1779780219.1548204,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":88576,"affinity_cache_ratio":0.9939404820683154,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":88576,"estimated_new_tokens":540,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":31161,"ongoing_decode_tokens":31161,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1818,"cache_hit":0,"new_prefill":89116,"score_linear":31161.0,"score_lmetric":89116},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":75693,"ongoing_decode_tokens":75693,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1976,"cache_hit":88576,"new_prefill":540,"score_linear":-12883.0,"score_lmetric":540},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2317,"cache_hit":0,"new_prefill":89116,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2213,"cache_hit":3584,"new_prefill":85532,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2751,"cache_hit":0,"new_prefill":89116,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1756,"cache_hit":3584,"new_prefill":85532,"score_linear":-3584.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":93596,"ongoing_decode_tokens":93596,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1710,"cache_hit":0,"new_prefill":89116,"score_linear":93596.0,"score_lmetric":89116},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":89116,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-12883.0,"chosen_score_lmetric":540,"t_first_token":349354.295658523,"t_first_token_unix":1779780219.3096023,"t_done":349354.91836789,"t_done_unix":1779780219.9323118},{"request_id":"1268861:17:1540278:951","session_id":"1268861","input_length":75693,"t_proxy_recv":349351.071372164,"t_decision_unix":1779780216.0853136,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":71168,"affinity_cache_ratio":0.9402190427120077,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":71168,"estimated_new_tokens":4525,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":31161,"ongoing_decode_tokens":31161,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1818,"cache_hit":0,"new_prefill":75693,"score_linear":31161.0,"score_lmetric":75693},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1976,"cache_hit":71168,"new_prefill":4525,"score_linear":-71168.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2317,"cache_hit":0,"new_prefill":75693,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2213,"cache_hit":0,"new_prefill":75693,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2751,"cache_hit":0,"new_prefill":75693,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1756,"cache_hit":0,"new_prefill":75693,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1710,"cache_hit":0,"new_prefill":75693,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":12288,"new_prefill":63405,"score_linear":-12288.0,"score_lmetric":0}],"chosen_score_linear":-71168.0,"chosen_score_lmetric":0,"t_first_token":349353.252419228,"t_first_token_unix":1779780218.2663624,"t_done":349355.000434677,"t_done_unix":1779780220.0143783},{"request_id":"1363943:33:1541065:952","session_id":"1363943","input_length":93596,"t_proxy_recv":349353.722290048,"t_decision_unix":1779780218.7362306,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":6,"chosen_idx":6,"affinity_cache_hit":93184,"affinity_cache_ratio":0.9955981024830121,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":93184,"estimated_new_tokens":412,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8006","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":31161,"ongoing_decode_tokens":31161,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1818,"cache_hit":8704,"new_prefill":84892,"score_linear":22457.0,"score_lmetric":84892},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":75693,"ongoing_decode_tokens":75693,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1976,"cache_hit":0,"new_prefill":93596,"score_linear":75693.0,"score_lmetric":93596},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2317,"cache_hit":0,"new_prefill":93596,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2213,"cache_hit":0,"new_prefill":93596,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2751,"cache_hit":4096,"new_prefill":89500,"score_linear":-4096.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1756,"cache_hit":0,"new_prefill":93596,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1710,"cache_hit":93184,"new_prefill":412,"score_linear":-93184.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":4096,"new_prefill":89500,"score_linear":-4096.0,"score_lmetric":0}],"chosen_score_linear":-93184.0,"chosen_score_lmetric":0,"t_first_token":349353.868913477,"t_first_token_unix":1779780218.8828573,"t_done":349360.3626448,"t_done_unix":1779780225.3765886},{"request_id":"1399948:23:1544108:954","session_id":"1399948","input_length":20031,"t_proxy_recv":349363.583074289,"t_decision_unix":1779780228.5970156,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":18432,"affinity_cache_ratio":0.9201737307173881,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":18432,"estimated_new_tokens":1599,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":31161,"ongoing_decode_tokens":31161,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1818,"cache_hit":0,"new_prefill":20031,"score_linear":31161.0,"score_lmetric":20031},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1986,"cache_hit":0,"new_prefill":20031,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2317,"cache_hit":18432,"new_prefill":1599,"score_linear":-18432.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2213,"cache_hit":0,"new_prefill":20031,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2751,"cache_hit":0,"new_prefill":20031,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1756,"cache_hit":0,"new_prefill":20031,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":20031,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":6144,"new_prefill":13887,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-18432.0,"chosen_score_lmetric":0,"t_first_token":349363.865597171,"t_first_token_unix":1779780228.8795407,"t_done":349364.306247268,"t_done_unix":1779780229.3201907},{"request_id":"1399948:24:1546940:955","session_id":"1399948","input_length":21809,"t_proxy_recv":349373.035628869,"t_decision_unix":1779780238.04957,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":19968,"affinity_cache_ratio":0.9155853088174607,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":19968,"estimated_new_tokens":1841,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":31161,"ongoing_decode_tokens":31161,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1818,"cache_hit":0,"new_prefill":21809,"score_linear":31161.0,"score_lmetric":21809},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1986,"cache_hit":0,"new_prefill":21809,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2320,"cache_hit":19968,"new_prefill":1841,"score_linear":-19968.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2213,"cache_hit":0,"new_prefill":21809,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2751,"cache_hit":0,"new_prefill":21809,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1756,"cache_hit":0,"new_prefill":21809,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":21809,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":6144,"new_prefill":15665,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-19968.0,"chosen_score_lmetric":0,"t_first_token":349373.383596479,"t_first_token_unix":1779780238.3975403,"t_done":349373.639504952,"t_done_unix":1779780238.653448},{"request_id":"1243831:17:1453633:883","session_id":"1243831","input_length":31161,"t_proxy_recv":349333.67914525,"t_decision_unix":1779780198.6930866,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":29184,"affinity_cache_ratio":0.9365553095215173,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":29184,"estimated_new_tokens":1977,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1818,"cache_hit":29184,"new_prefill":1977,"score_linear":-29184.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":158768,"ongoing_decode_tokens":87576,"pending_prefill_tokens":8216,"num_requests":2,"active_p_offloads":0,"cached_blocks":1937,"cache_hit":11264,"new_prefill":19897,"score_linear":147504.0,"score_lmetric":56226},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2317,"cache_hit":0,"new_prefill":31161,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2213,"cache_hit":0,"new_prefill":31161,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":71987,"ongoing_decode_tokens":71987,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2735,"cache_hit":0,"new_prefill":31161,"score_linear":71987.0,"score_lmetric":31161},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1756,"cache_hit":0,"new_prefill":31161,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1710,"cache_hit":0,"new_prefill":31161,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":31161,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-29184.0,"chosen_score_lmetric":0,"t_first_token":349334.065932641,"t_first_token_unix":1779780199.0798764,"t_done":349386.73627714,"t_done_unix":1779780251.7502234},{"request_id":"1399948:25:1553234:957","session_id":"1399948","input_length":22250,"t_proxy_recv":349393.56553987,"t_decision_unix":1779780258.5794816,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":21504,"affinity_cache_ratio":0.9664719101123596,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":21504,"estimated_new_tokens":746,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1822,"cache_hit":0,"new_prefill":22250,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":76017,"ongoing_decode_tokens":76017,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1986,"cache_hit":0,"new_prefill":22250,"score_linear":76017.0,"score_lmetric":22250},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2323,"cache_hit":21504,"new_prefill":746,"score_linear":-21504.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2213,"cache_hit":0,"new_prefill":22250,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2751,"cache_hit":0,"new_prefill":22250,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1756,"cache_hit":0,"new_prefill":22250,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":22250,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":6144,"new_prefill":16106,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-21504.0,"chosen_score_lmetric":0,"t_first_token":349393.716072719,"t_first_token_unix":1779780258.7300165,"t_done":349394.266292782,"t_done_unix":1779780259.2802362},{"request_id":"1268861:18:1553083:956","session_id":"1268861","input_length":76017,"t_proxy_recv":349393.128429849,"t_decision_unix":1779780258.1423712,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":75776,"affinity_cache_ratio":0.9968296565241985,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":75776,"estimated_new_tokens":241,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1822,"cache_hit":0,"new_prefill":76017,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1986,"cache_hit":75776,"new_prefill":241,"score_linear":-75776.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2323,"cache_hit":0,"new_prefill":76017,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2213,"cache_hit":0,"new_prefill":76017,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2751,"cache_hit":0,"new_prefill":76017,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1756,"cache_hit":0,"new_prefill":76017,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":76017,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":12288,"new_prefill":63729,"score_linear":-12288.0,"score_lmetric":0}],"chosen_score_linear":-75776.0,"chosen_score_lmetric":0,"t_first_token":349393.369895076,"t_first_token_unix":1779780258.3838384,"t_done":349394.409749975,"t_done_unix":1779780259.4236934},{"request_id":"1399948:26:1557859:959","session_id":"1399948","input_length":23042,"t_proxy_recv":349409.526914571,"t_decision_unix":1779780274.5408556,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":22016,"affinity_cache_ratio":0.9554726152243729,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":22016,"estimated_new_tokens":1026,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1822,"cache_hit":0,"new_prefill":23042,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1986,"cache_hit":0,"new_prefill":23042,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2324,"cache_hit":22016,"new_prefill":1026,"score_linear":-22016.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2213,"cache_hit":0,"new_prefill":23042,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2751,"cache_hit":0,"new_prefill":23042,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1756,"cache_hit":0,"new_prefill":23042,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":23042,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":6144,"new_prefill":16898,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-22016.0,"chosen_score_lmetric":0,"t_first_token":349409.735059769,"t_first_token_unix":1779780274.749003,"t_done":349410.021734465,"t_done_unix":1779780275.0356774},{"request_id":"1399948:27:1559017:960","session_id":"1399948","input_length":24139,"t_proxy_recv":349413.192295918,"t_decision_unix":1779780278.206237,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":23040,"affinity_cache_ratio":0.9544720162392808,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":23040,"estimated_new_tokens":1099,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1822,"cache_hit":0,"new_prefill":24139,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1986,"cache_hit":0,"new_prefill":24139,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2326,"cache_hit":23040,"new_prefill":1099,"score_linear":-23040.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2213,"cache_hit":0,"new_prefill":24139,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2751,"cache_hit":0,"new_prefill":24139,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1756,"cache_hit":0,"new_prefill":24139,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":24139,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":6144,"new_prefill":17995,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-23040.0,"chosen_score_lmetric":0,"t_first_token":349413.454869069,"t_first_token_unix":1779780278.4688127,"t_done":349414.423585161,"t_done_unix":1779780279.4375286},{"request_id":"1399948:28:1560859:961","session_id":"1399948","input_length":24386,"t_proxy_recv":349419.314081234,"t_decision_unix":1779780284.3280225,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":24064,"affinity_cache_ratio":0.9867957024522267,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":24064,"estimated_new_tokens":322,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1822,"cache_hit":0,"new_prefill":24386,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1986,"cache_hit":0,"new_prefill":24386,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2328,"cache_hit":24064,"new_prefill":322,"score_linear":-24064.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2213,"cache_hit":0,"new_prefill":24386,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2751,"cache_hit":0,"new_prefill":24386,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1756,"cache_hit":0,"new_prefill":24386,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":24386,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":6144,"new_prefill":18242,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-24064.0,"chosen_score_lmetric":0,"t_first_token":349419.394902698,"t_first_token_unix":1779780284.4088461,"t_done":349420.676691913,"t_done_unix":1779780285.6906352},{"request_id":"1399948:29:1571125:963","session_id":"1399948","input_length":28781,"t_proxy_recv":349453.182466934,"t_decision_unix":1779780318.1964078,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":24576,"affinity_cache_ratio":0.8538966679406553,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":24576,"estimated_new_tokens":4205,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":131413,"ongoing_decode_tokens":131413,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1822,"cache_hit":0,"new_prefill":28781,"score_linear":131413.0,"score_lmetric":28781},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1986,"cache_hit":0,"new_prefill":28781,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2329,"cache_hit":24576,"new_prefill":4205,"score_linear":-24576.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2213,"cache_hit":0,"new_prefill":28781,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2751,"cache_hit":0,"new_prefill":28781,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1756,"cache_hit":0,"new_prefill":28781,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":28781,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":6144,"new_prefill":22637,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-24576.0,"chosen_score_lmetric":0,"t_first_token":349454.066472325,"t_first_token_unix":1779780319.080416,"t_done":349454.340230824,"t_done_unix":1779780319.3541741},{"request_id":"1356885:6:1570780:962","session_id":"1356885","input_length":131413,"t_proxy_recv":349452.152442939,"t_decision_unix":1779780317.1663845,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":131072,"affinity_cache_ratio":0.9974051273466096,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":131072,"estimated_new_tokens":341,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1822,"cache_hit":131072,"new_prefill":341,"score_linear":-131072.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1986,"cache_hit":1024,"new_prefill":130389,"score_linear":-1024.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2329,"cache_hit":0,"new_prefill":131413,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2213,"cache_hit":0,"new_prefill":131413,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2751,"cache_hit":6656,"new_prefill":124757,"score_linear":-6656.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1756,"cache_hit":0,"new_prefill":131413,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":131413,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":131413,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-131072.0,"chosen_score_lmetric":0,"t_first_token":349452.78103858,"t_first_token_unix":1779780317.7949824,"t_done":349454.974392275,"t_done_unix":1779780319.9883356},{"request_id":"1270606:4:1578042:965","session_id":"1270606","input_length":109083,"t_proxy_recv":349476.806218424,"t_decision_unix":1779780341.8201592,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":108032,"affinity_cache_ratio":0.9903651348056067,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":108032,"estimated_new_tokens":1051,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1823,"cache_hit":0,"new_prefill":109083,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1986,"cache_hit":0,"new_prefill":109083,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2337,"cache_hit":0,"new_prefill":109083,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2213,"cache_hit":108032,"new_prefill":1051,"score_linear":-108032.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2751,"cache_hit":1536,"new_prefill":107547,"score_linear":-1536.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1756,"cache_hit":0,"new_prefill":109083,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":109083,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":109083,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-108032.0,"chosen_score_lmetric":0,"t_first_token":349477.574499781,"t_first_token_unix":1779780342.588443,"t_done":349477.978089052,"t_done_unix":1779780342.9920325},{"request_id":"1268861:19:1578134:966","session_id":"1268861","input_length":76963,"t_proxy_recv":349477.033538913,"t_decision_unix":1779780342.04748,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":75776,"affinity_cache_ratio":0.9845770045346465,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":75776,"estimated_new_tokens":1187,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1823,"cache_hit":0,"new_prefill":76963,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1986,"cache_hit":75776,"new_prefill":1187,"score_linear":-75776.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2337,"cache_hit":0,"new_prefill":76963,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":109083,"ongoing_decode_tokens":0,"pending_prefill_tokens":1051,"num_requests":1,"active_p_offloads":0,"cached_blocks":2213,"cache_hit":0,"new_prefill":76963,"score_linear":109083.0,"score_lmetric":78014},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2751,"cache_hit":0,"new_prefill":76963,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1756,"cache_hit":0,"new_prefill":76963,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":76963,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":12288,"new_prefill":64675,"score_linear":-12288.0,"score_lmetric":0}],"chosen_score_linear":-75776.0,"chosen_score_lmetric":0,"t_first_token":349477.587211435,"t_first_token_unix":1779780342.6011546,"t_done":349478.355898485,"t_done_unix":1779780343.3698423},{"request_id":"1399948:30:1578524:967","session_id":"1399948","input_length":30773,"t_proxy_recv":349478.372986792,"t_decision_unix":1779780343.3869286,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":28672,"affinity_cache_ratio":0.9317258635817113,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":28672,"estimated_new_tokens":2101,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1823,"cache_hit":0,"new_prefill":30773,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1988,"cache_hit":0,"new_prefill":30773,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2337,"cache_hit":28672,"new_prefill":2101,"score_linear":-28672.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2215,"cache_hit":0,"new_prefill":30773,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2751,"cache_hit":0,"new_prefill":30773,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1756,"cache_hit":0,"new_prefill":30773,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":30773,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":6144,"new_prefill":24629,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-28672.0,"chosen_score_lmetric":0,"t_first_token":349478.875487161,"t_first_token_unix":1779780343.8894312,"t_done":349479.180119094,"t_done_unix":1779780344.194063},{"request_id":"1399948:31:1579418:968","session_id":"1399948","input_length":30925,"t_proxy_recv":349481.599751135,"t_decision_unix":1779780346.613692,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":30720,"affinity_cache_ratio":0.9933710590137429,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":30720,"estimated_new_tokens":205,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1823,"cache_hit":0,"new_prefill":30925,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1988,"cache_hit":0,"new_prefill":30925,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2341,"cache_hit":30720,"new_prefill":205,"score_linear":-30720.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2215,"cache_hit":0,"new_prefill":30925,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2751,"cache_hit":0,"new_prefill":30925,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1756,"cache_hit":0,"new_prefill":30925,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":30925,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":6144,"new_prefill":24781,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-30720.0,"chosen_score_lmetric":0,"t_first_token":349481.694913973,"t_first_token_unix":1779780346.7088573,"t_done":349483.037703192,"t_done_unix":1779780348.0516467},{"request_id":"1356885:7:1580925:969","session_id":"1356885","input_length":131775,"t_proxy_recv":349486.91180343,"t_decision_unix":1779780351.925745,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":131584,"affinity_cache_ratio":0.9985505596660975,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":131584,"estimated_new_tokens":191,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1823,"cache_hit":131584,"new_prefill":191,"score_linear":-131584.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1988,"cache_hit":1024,"new_prefill":130751,"score_linear":-1024.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2341,"cache_hit":0,"new_prefill":131775,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2215,"cache_hit":0,"new_prefill":131775,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2751,"cache_hit":6656,"new_prefill":125119,"score_linear":-6656.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1756,"cache_hit":0,"new_prefill":131775,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":131775,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":131775,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-131584.0,"chosen_score_lmetric":0,"t_first_token":349487.326422841,"t_first_token_unix":1779780352.3403664,"t_done":349489.349882946,"t_done_unix":1779780354.3638265},{"request_id":"1399948:32:1581424:970","session_id":"1399948","input_length":31231,"t_proxy_recv":349488.349454112,"t_decision_unix":1779780353.363395,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":30720,"affinity_cache_ratio":0.9836380519355769,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":30720,"estimated_new_tokens":511,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":131775,"ongoing_decode_tokens":131775,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1823,"cache_hit":0,"new_prefill":31231,"score_linear":131775.0,"score_lmetric":31231},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1988,"cache_hit":0,"new_prefill":31231,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2341,"cache_hit":30720,"new_prefill":511,"score_linear":-30720.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2215,"cache_hit":0,"new_prefill":31231,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2751,"cache_hit":0,"new_prefill":31231,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1756,"cache_hit":0,"new_prefill":31231,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":31231,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":6144,"new_prefill":25087,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-30720.0,"chosen_score_lmetric":0,"t_first_token":349488.444387733,"t_first_token_unix":1779780353.4583309,"t_done":349489.456567098,"t_done_unix":1779780354.4705105},{"request_id":"1270606:5:1581455:971","session_id":"1270606","input_length":111387,"t_proxy_recv":349488.508732551,"t_decision_unix":1779780353.522674,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":109056,"affinity_cache_ratio":0.9790729618357618,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":109056,"estimated_new_tokens":2331,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":131775,"ongoing_decode_tokens":131775,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1823,"cache_hit":0,"new_prefill":111387,"score_linear":131775.0,"score_lmetric":111387},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1988,"cache_hit":0,"new_prefill":111387,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":31231,"ongoing_decode_tokens":31231,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2341,"cache_hit":0,"new_prefill":111387,"score_linear":31231.0,"score_lmetric":111387},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2215,"cache_hit":109056,"new_prefill":2331,"score_linear":-109056.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2751,"cache_hit":1536,"new_prefill":109851,"score_linear":-1536.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1756,"cache_hit":0,"new_prefill":111387,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":111387,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":111387,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-109056.0,"chosen_score_lmetric":0,"t_first_token":349490.260181978,"t_first_token_unix":1779780355.2741258,"t_done":349493.103075619,"t_done_unix":1779780358.1170192},{"request_id":"1399948:33:1583165:972","session_id":"1399948","input_length":31445,"t_proxy_recv":349494.258719225,"t_decision_unix":1779780359.27266,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":31232,"affinity_cache_ratio":0.9932262680871363,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":31232,"estimated_new_tokens":213,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1823,"cache_hit":0,"new_prefill":31445,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1988,"cache_hit":0,"new_prefill":31445,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2342,"cache_hit":31232,"new_prefill":213,"score_linear":-31232.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2220,"cache_hit":0,"new_prefill":31445,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2751,"cache_hit":0,"new_prefill":31445,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1756,"cache_hit":0,"new_prefill":31445,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":31445,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":6144,"new_prefill":25301,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-31232.0,"chosen_score_lmetric":0,"t_first_token":349494.34335927,"t_first_token_unix":1779780359.3573027,"t_done":349495.003792861,"t_done_unix":1779780360.0177362},{"request_id":"1277428:12:1586139:973","session_id":"1277428","input_length":47660,"t_proxy_recv":349504.398767146,"t_decision_unix":1779780369.4127085,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":46080,"affinity_cache_ratio":0.9668485102811583,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":46080,"estimated_new_tokens":1580,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1823,"cache_hit":0,"new_prefill":47660,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1988,"cache_hit":4608,"new_prefill":43052,"score_linear":-4608.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2342,"cache_hit":0,"new_prefill":47660,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2220,"cache_hit":17408,"new_prefill":30252,"score_linear":-17408.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2751,"cache_hit":0,"new_prefill":47660,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1756,"cache_hit":46080,"new_prefill":1580,"score_linear":-46080.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":47660,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":47660,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-46080.0,"chosen_score_lmetric":0,"t_first_token":349504.864990705,"t_first_token_unix":1779780369.878934,"t_done":349505.47968611,"t_done_unix":1779780370.4936297},{"request_id":"1399948:34:1586232:974","session_id":"1399948","input_length":32143,"t_proxy_recv":349504.748938458,"t_decision_unix":1779780369.7628798,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":31232,"affinity_cache_ratio":0.97165790374265,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":31232,"estimated_new_tokens":911,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1823,"cache_hit":0,"new_prefill":32143,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1988,"cache_hit":0,"new_prefill":32143,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2342,"cache_hit":31232,"new_prefill":911,"score_linear":-31232.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2220,"cache_hit":0,"new_prefill":32143,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2751,"cache_hit":0,"new_prefill":32143,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":47660,"ongoing_decode_tokens":0,"pending_prefill_tokens":1580,"num_requests":1,"active_p_offloads":0,"cached_blocks":1756,"cache_hit":0,"new_prefill":32143,"score_linear":47660.0,"score_lmetric":33723},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":32143,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":6144,"new_prefill":25999,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-31232.0,"chosen_score_lmetric":0,"t_first_token":349504.953906021,"t_first_token_unix":1779780369.9678493,"t_done":349505.861797132,"t_done_unix":1779780370.8757405},{"request_id":"1294611:15:1590202:975","session_id":"1294611","input_length":75249,"t_proxy_recv":349517.936677102,"t_decision_unix":1779780382.9506183,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":72192,"affinity_cache_ratio":0.9593748754136268,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":72192,"estimated_new_tokens":3057,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1823,"cache_hit":0,"new_prefill":75249,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1988,"cache_hit":0,"new_prefill":75249,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2344,"cache_hit":0,"new_prefill":75249,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2220,"cache_hit":0,"new_prefill":75249,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2751,"cache_hit":72192,"new_prefill":3057,"score_linear":-72192.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1759,"cache_hit":12800,"new_prefill":62449,"score_linear":-12800.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":75249,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":12800,"new_prefill":62449,"score_linear":-12800.0,"score_lmetric":0}],"chosen_score_linear":-72192.0,"chosen_score_lmetric":0,"t_first_token":349519.437017431,"t_first_token_unix":1779780384.4509609,"t_done":349522.155180078,"t_done_unix":1779780387.1691232},{"request_id":"1277428:13:1593499:976","session_id":"1277428","input_length":49316,"t_proxy_recv":349529.587854603,"t_decision_unix":1779780394.601796,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":47616,"affinity_cache_ratio":0.9655284289074539,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":47616,"estimated_new_tokens":1700,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1823,"cache_hit":0,"new_prefill":49316,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1988,"cache_hit":4608,"new_prefill":44708,"score_linear":-4608.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2344,"cache_hit":0,"new_prefill":49316,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2220,"cache_hit":17408,"new_prefill":31908,"score_linear":-17408.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2757,"cache_hit":0,"new_prefill":49316,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1759,"cache_hit":47616,"new_prefill":1700,"score_linear":-47616.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":49316,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":49316,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-47616.0,"chosen_score_lmetric":0,"t_first_token":349530.240835371,"t_first_token_unix":1779780395.254779,"t_done":349531.770781131,"t_done_unix":1779780396.7847242},{"request_id":"1294611:15:1594038:977","session_id":"1294611","input_length":75584,"t_proxy_recv":349531.316487967,"t_decision_unix":1779780396.330429,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":75264,"affinity_cache_ratio":0.995766299745978,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":75264,"estimated_new_tokens":320,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1823,"cache_hit":0,"new_prefill":75584,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1988,"cache_hit":0,"new_prefill":75584,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2344,"cache_hit":0,"new_prefill":75584,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2220,"cache_hit":0,"new_prefill":75584,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2757,"cache_hit":75264,"new_prefill":320,"score_linear":-75264.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":49316,"ongoing_decode_tokens":49316,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1759,"cache_hit":12800,"new_prefill":62784,"score_linear":36516.0,"score_lmetric":62784},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":75584,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":12800,"new_prefill":62784,"score_linear":-12800.0,"score_lmetric":0}],"chosen_score_linear":-75264.0,"chosen_score_lmetric":0,"t_first_token":349531.438578143,"t_first_token_unix":1779780396.4525218,"t_done":349532.551646083,"t_done_unix":1779780397.5655894},{"request_id":"1268861:20:1595141:978","session_id":"1268861","input_length":77218,"t_proxy_recv":349535.281580833,"t_decision_unix":1779780400.295522,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":76800,"affinity_cache_ratio":0.9945867543836929,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":76800,"estimated_new_tokens":418,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1823,"cache_hit":0,"new_prefill":77218,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1988,"cache_hit":76800,"new_prefill":418,"score_linear":-76800.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2344,"cache_hit":0,"new_prefill":77218,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2220,"cache_hit":0,"new_prefill":77218,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2757,"cache_hit":0,"new_prefill":77218,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":0,"new_prefill":77218,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":77218,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":12288,"new_prefill":64930,"score_linear":-12288.0,"score_lmetric":0}],"chosen_score_linear":-76800.0,"chosen_score_lmetric":0,"t_first_token":349535.531205904,"t_first_token_unix":1779780400.5451498,"t_done":349536.067263131,"t_done_unix":1779780401.0812068},{"request_id":"1399948:35:1602048:980","session_id":"1399948","input_length":33533,"t_proxy_recv":349558.651049957,"t_decision_unix":1779780423.6649907,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":32256,"affinity_cache_ratio":0.9619181105179972,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":32256,"estimated_new_tokens":1277,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1823,"cache_hit":0,"new_prefill":33533,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1988,"cache_hit":0,"new_prefill":33533,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2344,"cache_hit":32256,"new_prefill":1277,"score_linear":-32256.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2220,"cache_hit":0,"new_prefill":33533,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":75716,"ongoing_decode_tokens":75716,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2757,"cache_hit":0,"new_prefill":33533,"score_linear":75716.0,"score_lmetric":33533},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":0,"new_prefill":33533,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":33533,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":6144,"new_prefill":27389,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-32256.0,"chosen_score_lmetric":0,"t_first_token":349559.056442685,"t_first_token_unix":1779780424.0703864,"t_done":349560.079805617,"t_done_unix":1779780425.0937498},{"request_id":"1253804:9:1605110:981","session_id":"1253804","input_length":90656,"t_proxy_recv":349568.641908175,"t_decision_unix":1779780433.6558495,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":89088,"affinity_cache_ratio":0.9827038475114719,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":89088,"estimated_new_tokens":1568,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1823,"cache_hit":0,"new_prefill":90656,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1988,"cache_hit":89088,"new_prefill":1568,"score_linear":-89088.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2346,"cache_hit":0,"new_prefill":90656,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2220,"cache_hit":3584,"new_prefill":87072,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":75716,"ongoing_decode_tokens":75716,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2757,"cache_hit":0,"new_prefill":90656,"score_linear":75716.0,"score_lmetric":90656},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":3584,"new_prefill":87072,"score_linear":-3584.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":90656,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":90656,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-89088.0,"chosen_score_lmetric":0,"t_first_token":349569.611011656,"t_first_token_unix":1779780434.6249557,"t_done":349573.244014864,"t_done_unix":1779780438.2579582},{"request_id":"1294611:15:1596163:979","session_id":"1294611","input_length":75716,"t_proxy_recv":349538.462417695,"t_decision_unix":1779780403.476359,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":75264,"affinity_cache_ratio":0.9940303238417243,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":75264,"estimated_new_tokens":452,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1823,"cache_hit":0,"new_prefill":75716,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1988,"cache_hit":0,"new_prefill":75716,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2344,"cache_hit":0,"new_prefill":75716,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2220,"cache_hit":0,"new_prefill":75716,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2757,"cache_hit":75264,"new_prefill":452,"score_linear":-75264.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":12800,"new_prefill":62916,"score_linear":-12800.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":75716,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":12800,"new_prefill":62916,"score_linear":-12800.0,"score_lmetric":0}],"chosen_score_linear":-75264.0,"chosen_score_lmetric":0,"t_first_token":349538.608837854,"t_first_token_unix":1779780403.6227813,"t_done":349577.034664421,"t_done_unix":1779780442.048608},{"request_id":"1268861:21:1609831:982","session_id":"1268861","input_length":77293,"t_proxy_recv":349584.388720826,"t_decision_unix":1779780449.402662,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":76800,"affinity_cache_ratio":0.9936216733727504,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":76800,"estimated_new_tokens":493,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1823,"cache_hit":0,"new_prefill":77293,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1991,"cache_hit":76800,"new_prefill":493,"score_linear":-76800.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2346,"cache_hit":0,"new_prefill":77293,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2220,"cache_hit":0,"new_prefill":77293,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2759,"cache_hit":0,"new_prefill":77293,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":0,"new_prefill":77293,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":77293,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":12288,"new_prefill":65005,"score_linear":-12288.0,"score_lmetric":0}],"chosen_score_linear":-76800.0,"chosen_score_lmetric":0,"t_first_token":349584.493924329,"t_first_token_unix":1779780449.5078676,"t_done":349586.104353566,"t_done_unix":1779780451.1182966},{"request_id":"1253804:10:1611505:983","session_id":"1253804","input_length":91103,"t_proxy_recv":349589.864770732,"t_decision_unix":1779780454.8787122,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":90624,"affinity_cache_ratio":0.9947422148557128,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":90624,"estimated_new_tokens":479,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1823,"cache_hit":0,"new_prefill":91103,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1992,"cache_hit":90624,"new_prefill":479,"score_linear":-90624.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2346,"cache_hit":0,"new_prefill":91103,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2220,"cache_hit":3584,"new_prefill":87519,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2759,"cache_hit":0,"new_prefill":91103,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":3584,"new_prefill":87519,"score_linear":-3584.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":91103,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":91103,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-90624.0,"chosen_score_lmetric":0,"t_first_token":349590.031985754,"t_first_token_unix":1779780455.0459292,"t_done":349590.1662335,"t_done_unix":1779780455.180177},{"request_id":"1399948:36:1612467:984","session_id":"1399948","input_length":34773,"t_proxy_recv":349592.969978812,"t_decision_unix":1779780457.98392,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":33280,"affinity_cache_ratio":0.9570643890374716,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":33280,"estimated_new_tokens":1493,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1823,"cache_hit":0,"new_prefill":34773,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1992,"cache_hit":0,"new_prefill":34773,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2346,"cache_hit":33280,"new_prefill":1493,"score_linear":-33280.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2220,"cache_hit":0,"new_prefill":34773,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2759,"cache_hit":0,"new_prefill":34773,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":0,"new_prefill":34773,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":34773,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":6144,"new_prefill":28629,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-33280.0,"chosen_score_lmetric":0,"t_first_token":349593.318529441,"t_first_token_unix":1779780458.3324733,"t_done":349593.872165917,"t_done_unix":1779780458.88611},{"request_id":"1253804:11:1613030:985","session_id":"1253804","input_length":91131,"t_proxy_recv":349595.199260007,"t_decision_unix":1779780460.213201,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":90624,"affinity_cache_ratio":0.9944365803074695,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":90624,"estimated_new_tokens":507,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1823,"cache_hit":0,"new_prefill":91131,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1992,"cache_hit":90624,"new_prefill":507,"score_linear":-90624.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2349,"cache_hit":0,"new_prefill":91131,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2220,"cache_hit":3584,"new_prefill":87547,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2759,"cache_hit":0,"new_prefill":91131,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":3584,"new_prefill":87547,"score_linear":-3584.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":91131,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":91131,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-90624.0,"chosen_score_lmetric":0,"t_first_token":349595.317418054,"t_first_token_unix":1779780460.331361,"t_done":349597.406196808,"t_done_unix":1779780462.42014},{"request_id":"1399948:37:1613628:986","session_id":"1399948","input_length":35308,"t_proxy_recv":349597.050567136,"t_decision_unix":1779780462.0645087,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":34816,"affinity_cache_ratio":0.9860654809108418,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":34816,"estimated_new_tokens":492,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1823,"cache_hit":0,"new_prefill":35308,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":91131,"ongoing_decode_tokens":91131,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1992,"cache_hit":0,"new_prefill":35308,"score_linear":91131.0,"score_lmetric":35308},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2349,"cache_hit":34816,"new_prefill":492,"score_linear":-34816.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2220,"cache_hit":0,"new_prefill":35308,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2759,"cache_hit":0,"new_prefill":35308,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":0,"new_prefill":35308,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":35308,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":6144,"new_prefill":29164,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-34816.0,"chosen_score_lmetric":0,"t_first_token":349597.261562533,"t_first_token_unix":1779780462.2755063,"t_done":349598.409466993,"t_done_unix":1779780463.4234102},{"request_id":"1399948:38:1616890:987","session_id":"1399948","input_length":35581,"t_proxy_recv":349608.233392795,"t_decision_unix":1779780473.247334,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":35328,"affinity_cache_ratio":0.9928894634776988,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":35328,"estimated_new_tokens":253,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1823,"cache_hit":0,"new_prefill":35581,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1993,"cache_hit":0,"new_prefill":35581,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2350,"cache_hit":35328,"new_prefill":253,"score_linear":-35328.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2220,"cache_hit":0,"new_prefill":35581,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2759,"cache_hit":0,"new_prefill":35581,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":0,"new_prefill":35581,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":35581,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":6144,"new_prefill":29437,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-35328.0,"chosen_score_lmetric":0,"t_first_token":349608.338089245,"t_first_token_unix":1779780473.3520322,"t_done":349608.862983381,"t_done_unix":1779780473.8769264},{"request_id":"1399948:39:1618250:989","session_id":"1399948","input_length":35682,"t_proxy_recv":349612.92298961,"t_decision_unix":1779780477.9369311,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":35328,"affinity_cache_ratio":0.9900790314444258,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":35328,"estimated_new_tokens":354,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1823,"cache_hit":0,"new_prefill":35682,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1993,"cache_hit":0,"new_prefill":35682,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2350,"cache_hit":35328,"new_prefill":354,"score_linear":-35328.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2220,"cache_hit":0,"new_prefill":35682,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2759,"cache_hit":0,"new_prefill":35682,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":0,"new_prefill":35682,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":35682,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":6144,"new_prefill":29538,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-35328.0,"chosen_score_lmetric":0,"t_first_token":349612.990831155,"t_first_token_unix":1779780478.0047746,"t_done":349613.438207779,"t_done_unix":1779780478.452151},{"request_id":"1399948:40:1620970:991","session_id":"1399948","input_length":35958,"t_proxy_recv":349621.910204531,"t_decision_unix":1779780486.9241462,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":35328,"affinity_cache_ratio":0.9824795594860671,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":35328,"estimated_new_tokens":630,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1823,"cache_hit":0,"new_prefill":35958,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":78439,"ongoing_decode_tokens":0,"pending_prefill_tokens":1127,"num_requests":1,"active_p_offloads":0,"cached_blocks":1993,"cache_hit":0,"new_prefill":35958,"score_linear":78439.0,"score_lmetric":37085},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2350,"cache_hit":35328,"new_prefill":630,"score_linear":-35328.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2220,"cache_hit":0,"new_prefill":35958,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2759,"cache_hit":0,"new_prefill":35958,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":0,"new_prefill":35958,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":35958,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":6144,"new_prefill":29814,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-35328.0,"chosen_score_lmetric":0,"t_first_token":349622.044597119,"t_first_token_unix":1779780487.0585413,"t_done":349622.414636026,"t_done_unix":1779780487.4285793},{"request_id":"1268861:22:1620830:990","session_id":"1268861","input_length":78439,"t_proxy_recv":349621.45279796,"t_decision_unix":1779780486.4667394,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":77312,"affinity_cache_ratio":0.9856321472736777,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":77312,"estimated_new_tokens":1127,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1823,"cache_hit":0,"new_prefill":78439,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1993,"cache_hit":77312,"new_prefill":1127,"score_linear":-77312.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2350,"cache_hit":0,"new_prefill":78439,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2220,"cache_hit":0,"new_prefill":78439,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2759,"cache_hit":0,"new_prefill":78439,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":0,"new_prefill":78439,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":78439,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":12288,"new_prefill":66151,"score_linear":-12288.0,"score_lmetric":0}],"chosen_score_linear":-77312.0,"chosen_score_lmetric":0,"t_first_token":349622.150656896,"t_first_token_unix":1779780487.1646006,"t_done":349623.4312699,"t_done_unix":1779780488.445213},{"request_id":"1253804:12:1621957:992","session_id":"1253804","input_length":91389,"t_proxy_recv":349625.357924895,"t_decision_unix":1779780490.3718665,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":91136,"affinity_cache_ratio":0.9972316143080677,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":91136,"estimated_new_tokens":253,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1823,"cache_hit":0,"new_prefill":91389,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1995,"cache_hit":91136,"new_prefill":253,"score_linear":-91136.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2351,"cache_hit":0,"new_prefill":91389,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2220,"cache_hit":3584,"new_prefill":87805,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2759,"cache_hit":0,"new_prefill":91389,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":3584,"new_prefill":87805,"score_linear":-3584.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":91389,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":91389,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-91136.0,"chosen_score_lmetric":0,"t_first_token":349625.535124959,"t_first_token_unix":1779780490.5490682,"t_done":349625.831145203,"t_done_unix":1779780490.8450887},{"request_id":"1399948:41:1624143:993","session_id":"1399948","input_length":36168,"t_proxy_recv":349632.717171757,"t_decision_unix":1779780497.731113,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":35840,"affinity_cache_ratio":0.990931209909312,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":35840,"estimated_new_tokens":328,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1823,"cache_hit":0,"new_prefill":36168,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1995,"cache_hit":0,"new_prefill":36168,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2351,"cache_hit":35840,"new_prefill":328,"score_linear":-35840.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2220,"cache_hit":0,"new_prefill":36168,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2759,"cache_hit":0,"new_prefill":36168,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":0,"new_prefill":36168,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":36168,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":6144,"new_prefill":30024,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-35840.0,"chosen_score_lmetric":0,"t_first_token":349632.850567343,"t_first_token_unix":1779780497.8645108,"t_done":349633.523333125,"t_done_unix":1779780498.5372765},{"request_id":"1253804:13:1624613:994","session_id":"1253804","input_length":91440,"t_proxy_recv":349634.302620024,"t_decision_unix":1779780499.3165615,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":91136,"affinity_cache_ratio":0.9966754155730534,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":91136,"estimated_new_tokens":304,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1823,"cache_hit":0,"new_prefill":91440,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1995,"cache_hit":91136,"new_prefill":304,"score_linear":-91136.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2351,"cache_hit":0,"new_prefill":91440,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2220,"cache_hit":3584,"new_prefill":87856,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2759,"cache_hit":0,"new_prefill":91440,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":3584,"new_prefill":87856,"score_linear":-3584.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":91440,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":91440,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-91136.0,"chosen_score_lmetric":0,"t_first_token":349634.429715274,"t_first_token_unix":1779780499.4436588,"t_done":349635.436244744,"t_done_unix":1779780500.4501877},{"request_id":"1253804:14:1626575:995","session_id":"1253804","input_length":92367,"t_proxy_recv":349640.811847776,"t_decision_unix":1779780505.825789,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":91136,"affinity_cache_ratio":0.9866727294380028,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":91136,"estimated_new_tokens":1231,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1823,"cache_hit":0,"new_prefill":92367,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1995,"cache_hit":91136,"new_prefill":1231,"score_linear":-91136.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2351,"cache_hit":0,"new_prefill":92367,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2220,"cache_hit":3584,"new_prefill":88783,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2759,"cache_hit":0,"new_prefill":92367,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":3584,"new_prefill":88783,"score_linear":-3584.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":92367,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":92367,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-91136.0,"chosen_score_lmetric":0,"t_first_token":349641.443887307,"t_first_token_unix":1779780506.457831,"t_done":349642.256897372,"t_done_unix":1779780507.2708411},{"request_id":"1253804:15:1628485:997","session_id":"1253804","input_length":92701,"t_proxy_recv":349647.138918913,"t_decision_unix":1779780512.1528602,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":92160,"affinity_cache_ratio":0.9941640327504557,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":92160,"estimated_new_tokens":541,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1823,"cache_hit":0,"new_prefill":92701,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1997,"cache_hit":92160,"new_prefill":541,"score_linear":-92160.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":36984,"ongoing_decode_tokens":0,"pending_prefill_tokens":1144,"num_requests":1,"active_p_offloads":0,"cached_blocks":2351,"cache_hit":0,"new_prefill":92701,"score_linear":36984.0,"score_lmetric":93845},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2220,"cache_hit":3584,"new_prefill":89117,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2759,"cache_hit":0,"new_prefill":92701,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":3584,"new_prefill":89117,"score_linear":-3584.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":92701,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":92701,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-92160.0,"chosen_score_lmetric":0,"t_first_token":349647.423847991,"t_first_token_unix":1779780512.4377913,"t_done":349647.806201751,"t_done_unix":1779780512.820145},{"request_id":"1399948:42:1628429:996","session_id":"1399948","input_length":36984,"t_proxy_recv":349646.916031299,"t_decision_unix":1779780511.9299724,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":35840,"affinity_cache_ratio":0.9690677049534934,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":35840,"estimated_new_tokens":1144,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1823,"cache_hit":0,"new_prefill":36984,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1997,"cache_hit":0,"new_prefill":36984,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2351,"cache_hit":35840,"new_prefill":1144,"score_linear":-35840.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2220,"cache_hit":0,"new_prefill":36984,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2759,"cache_hit":0,"new_prefill":36984,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":0,"new_prefill":36984,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":36984,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":6144,"new_prefill":30840,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-35840.0,"chosen_score_lmetric":0,"t_first_token":349647.202453414,"t_first_token_unix":1779780512.2163966,"t_done":349648.422652465,"t_done_unix":1779780513.4365954},{"request_id":"1294611:15:1631116:1000","session_id":"1294611","input_length":79890,"t_proxy_recv":349655.770460768,"t_decision_unix":1779780520.784402,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":79360,"affinity_cache_ratio":0.9933658780823632,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":79360,"estimated_new_tokens":530,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1823,"cache_hit":0,"new_prefill":79890,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":172027,"ongoing_decode_tokens":0,"pending_prefill_tokens":3579,"num_requests":2,"active_p_offloads":0,"cached_blocks":1998,"cache_hit":0,"new_prefill":79890,"score_linear":172027.0,"score_lmetric":166938},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2353,"cache_hit":0,"new_prefill":79890,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2220,"cache_hit":0,"new_prefill":79890,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2759,"cache_hit":79360,"new_prefill":530,"score_linear":-79360.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":12800,"new_prefill":67090,"score_linear":-12800.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":79890,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":12800,"new_prefill":67090,"score_linear":-12800.0,"score_lmetric":0}],"chosen_score_linear":-79360.0,"chosen_score_lmetric":0,"t_first_token":349655.949190066,"t_first_token_unix":1779780520.9631333,"t_done":349658.702797297,"t_done_unix":1779780523.7167408},{"request_id":"1294611:15:1634779:1004","session_id":"1294611","input_length":80221,"t_proxy_recv":349667.8845573,"t_decision_unix":1779780532.8984985,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":79872,"affinity_cache_ratio":0.995649518205956,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":79872,"estimated_new_tokens":349,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1823,"cache_hit":0,"new_prefill":80221,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":251720,"ongoing_decode_tokens":0,"pending_prefill_tokens":4936,"num_requests":3,"active_p_offloads":0,"cached_blocks":1998,"cache_hit":0,"new_prefill":80221,"score_linear":251720.0,"score_lmetric":255471},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2353,"cache_hit":0,"new_prefill":80221,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2220,"cache_hit":0,"new_prefill":80221,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2760,"cache_hit":79872,"new_prefill":349,"score_linear":-79872.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":12800,"new_prefill":67421,"score_linear":-12800.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":80221,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":12800,"new_prefill":67421,"score_linear":-12800.0,"score_lmetric":0}],"chosen_score_linear":-79872.0,"chosen_score_lmetric":0,"t_first_token":349668.022883921,"t_first_token_unix":1779780533.036827,"t_done":349668.581830409,"t_done_unix":1779780533.5957735},{"request_id":"1388507:3:1629072:998","session_id":"1388507","input_length":79074,"t_proxy_recv":349649.224129271,"t_decision_unix":1779780514.2380705,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":75776,"affinity_cache_ratio":0.9582922325922554,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":75776,"estimated_new_tokens":3298,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1823,"cache_hit":0,"new_prefill":79074,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1998,"cache_hit":75776,"new_prefill":3298,"score_linear":-75776.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2353,"cache_hit":0,"new_prefill":79074,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2220,"cache_hit":0,"new_prefill":79074,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2759,"cache_hit":0,"new_prefill":79074,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":0,"new_prefill":79074,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":79074,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":79074,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-75776.0,"chosen_score_lmetric":0,"t_first_token":349670.517933447,"t_first_token_unix":1779780535.531877,"t_done":349672.270828442,"t_done_unix":1779780537.284772},{"request_id":"1268861:23:1631587:1001","session_id":"1268861","input_length":79693,"t_proxy_recv":349657.183905943,"t_decision_unix":1779780522.1978474,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":78336,"affinity_cache_ratio":0.9829721556472965,"affinity_num_requests":2,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":78336,"estimated_new_tokens":1357,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1823,"cache_hit":0,"new_prefill":79693,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":172027,"ongoing_decode_tokens":0,"pending_prefill_tokens":3579,"num_requests":2,"active_p_offloads":0,"cached_blocks":1998,"cache_hit":78336,"new_prefill":1357,"score_linear":93691.0,"score_lmetric":9872},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2353,"cache_hit":0,"new_prefill":79693,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2220,"cache_hit":0,"new_prefill":79693,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":79890,"ongoing_decode_tokens":79890,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2759,"cache_hit":0,"new_prefill":79693,"score_linear":79890.0,"score_lmetric":79693},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":0,"new_prefill":79693,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":79693,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":12288,"new_prefill":67405,"score_linear":-12288.0,"score_lmetric":0}],"chosen_score_linear":93691.0,"chosen_score_lmetric":9872,"t_first_token":349671.06337942,"t_first_token_unix":1779780536.077323,"t_done":349672.419712291,"t_done_unix":1779780537.4336555},{"request_id":"1253804:16:1629713:999","session_id":"1253804","input_length":92953,"t_proxy_recv":349651.186510241,"t_decision_unix":1779780516.2004516,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":92672,"affinity_cache_ratio":0.9969769668542167,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":92672,"estimated_new_tokens":281,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1823,"cache_hit":0,"new_prefill":92953,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":79074,"ongoing_decode_tokens":0,"pending_prefill_tokens":3298,"num_requests":1,"active_p_offloads":0,"cached_blocks":1998,"cache_hit":92672,"new_prefill":281,"score_linear":-13598.0,"score_lmetric":3579},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2353,"cache_hit":0,"new_prefill":92953,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2220,"cache_hit":3584,"new_prefill":89369,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2759,"cache_hit":0,"new_prefill":92953,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":3584,"new_prefill":89369,"score_linear":-3584.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":92953,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":92953,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-13598.0,"chosen_score_lmetric":3579,"t_first_token":349670.525554524,"t_first_token_unix":1779780535.5394976,"t_done":349672.695867977,"t_done_unix":1779780537.7098114},{"request_id":"1253804:17:1631602:1002","session_id":"1253804","input_length":93219,"t_proxy_recv":349672.710995759,"t_decision_unix":1779780537.7249372,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":92672,"affinity_cache_ratio":0.9941320975337645,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":92672,"estimated_new_tokens":547,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1823,"cache_hit":0,"new_prefill":93219,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2006,"cache_hit":92672,"new_prefill":547,"score_linear":-92672.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2353,"cache_hit":0,"new_prefill":93219,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2220,"cache_hit":3584,"new_prefill":89635,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2760,"cache_hit":0,"new_prefill":93219,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":3584,"new_prefill":89635,"score_linear":-3584.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":93219,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":93219,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-92672.0,"chosen_score_lmetric":0,"t_first_token":349672.993660217,"t_first_token_unix":1779780538.0076041,"t_done":349673.764285551,"t_done_unix":1779780538.778229},{"request_id":"1253804:18:1633257:1003","session_id":"1253804","input_length":93509,"t_proxy_recv":349673.779512144,"t_decision_unix":1779780538.793454,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":93184,"affinity_cache_ratio":0.9965243987209788,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":93184,"estimated_new_tokens":325,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1823,"cache_hit":0,"new_prefill":93509,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2007,"cache_hit":93184,"new_prefill":325,"score_linear":-93184.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2353,"cache_hit":0,"new_prefill":93509,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2220,"cache_hit":3584,"new_prefill":89925,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2760,"cache_hit":0,"new_prefill":93509,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":3584,"new_prefill":89925,"score_linear":-3584.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":93509,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":93509,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-93184.0,"chosen_score_lmetric":0,"t_first_token":349674.062535105,"t_first_token_unix":1779780539.0764787,"t_done":349674.958174857,"t_done_unix":1779780539.9721189},{"request_id":"1253804:19:1634903:1005","session_id":"1253804","input_length":93664,"t_proxy_recv":349674.975018824,"t_decision_unix":1779780539.9889603,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":93184,"affinity_cache_ratio":0.9948752989408951,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":93184,"estimated_new_tokens":480,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1823,"cache_hit":0,"new_prefill":93664,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2007,"cache_hit":93184,"new_prefill":480,"score_linear":-93184.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2353,"cache_hit":0,"new_prefill":93664,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2220,"cache_hit":3584,"new_prefill":90080,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":80717,"ongoing_decode_tokens":80717,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2760,"cache_hit":0,"new_prefill":93664,"score_linear":80717.0,"score_lmetric":93664},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":3584,"new_prefill":90080,"score_linear":-3584.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":93664,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":93664,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-93184.0,"chosen_score_lmetric":0,"t_first_token":349675.149329429,"t_first_token_unix":1779780540.1632736,"t_done":349679.183275065,"t_done_unix":1779780544.1972184},{"request_id":"1253804:20:1638889:1007","session_id":"1253804","input_length":94111,"t_proxy_recv":349681.430701972,"t_decision_unix":1779780546.4446433,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":93696,"affinity_cache_ratio":0.9955903135658956,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":93696,"estimated_new_tokens":415,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1823,"cache_hit":0,"new_prefill":94111,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2008,"cache_hit":93696,"new_prefill":415,"score_linear":-93696.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2353,"cache_hit":0,"new_prefill":94111,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2220,"cache_hit":3584,"new_prefill":90527,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":80717,"ongoing_decode_tokens":80717,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2760,"cache_hit":0,"new_prefill":94111,"score_linear":80717.0,"score_lmetric":94111},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":3584,"new_prefill":90527,"score_linear":-3584.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":94111,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":94111,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-93696.0,"chosen_score_lmetric":0,"t_first_token":349681.609470895,"t_first_token_unix":1779780546.6234143,"t_done":349682.441684226,"t_done_unix":1779780547.4556277},{"request_id":"1258908:2:1639411:1008","session_id":"1258908","input_length":9804,"t_proxy_recv":349683.232236575,"t_decision_unix":1779780548.2461782,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":9216,"affinity_cache_ratio":0.9400244798041616,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":9216,"estimated_new_tokens":588,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1823,"cache_hit":0,"new_prefill":9804,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2008,"cache_hit":0,"new_prefill":9804,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2353,"cache_hit":0,"new_prefill":9804,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2220,"cache_hit":9216,"new_prefill":588,"score_linear":-9216.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":80717,"ongoing_decode_tokens":80717,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2760,"cache_hit":0,"new_prefill":9804,"score_linear":80717.0,"score_lmetric":9804},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":0,"new_prefill":9804,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":9804,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":9804,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-9216.0,"chosen_score_lmetric":0,"t_first_token":349683.987395852,"t_first_token_unix":1779780549.00134,"t_done":349685.560516922,"t_done_unix":1779780550.5744598},{"request_id":"1294611:15:1636645:1006","session_id":"1294611","input_length":80717,"t_proxy_recv":349673.921707281,"t_decision_unix":1779780538.9356396,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":79872,"affinity_cache_ratio":0.9895313254952488,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":79872,"estimated_new_tokens":845,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1823,"cache_hit":0,"new_prefill":80717,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":93509,"ongoing_decode_tokens":0,"pending_prefill_tokens":325,"num_requests":1,"active_p_offloads":0,"cached_blocks":2007,"cache_hit":0,"new_prefill":80717,"score_linear":93509.0,"score_lmetric":81042},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2353,"cache_hit":0,"new_prefill":80717,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2220,"cache_hit":0,"new_prefill":80717,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2760,"cache_hit":79872,"new_prefill":845,"score_linear":-79872.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":12800,"new_prefill":67917,"score_linear":-12800.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":80717,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":12800,"new_prefill":67917,"score_linear":-12800.0,"score_lmetric":0}],"chosen_score_linear":-79872.0,"chosen_score_lmetric":0,"t_first_token":349674.328409795,"t_first_token_unix":1779780539.3423533,"t_done":349697.539298806,"t_done_unix":1779780562.5532422},{"request_id":"1399948:43:1643413:1010","session_id":"1399948","input_length":38102,"t_proxy_recv":349696.709371856,"t_decision_unix":1779780561.723313,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":36864,"affinity_cache_ratio":0.9675082672825573,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":36864,"estimated_new_tokens":1238,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1823,"cache_hit":0,"new_prefill":38102,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":94269,"ongoing_decode_tokens":94269,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2008,"cache_hit":0,"new_prefill":38102,"score_linear":94269.0,"score_lmetric":38102},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2353,"cache_hit":36864,"new_prefill":1238,"score_linear":-36864.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":0,"new_prefill":38102,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":80717,"ongoing_decode_tokens":80717,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2760,"cache_hit":0,"new_prefill":38102,"score_linear":80717.0,"score_lmetric":38102},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":0,"new_prefill":38102,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":38102,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":6144,"new_prefill":31958,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-36864.0,"chosen_score_lmetric":0,"t_first_token":349697.083952584,"t_first_token_unix":1779780562.0978956,"t_done":349697.873435948,"t_done_unix":1779780562.8873792},{"request_id":"1268861:24:1643502:1011","session_id":"1268861","input_length":80019,"t_proxy_recv":349697.058120102,"t_decision_unix":1779780562.0720615,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":79360,"affinity_cache_ratio":0.9917644559417138,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":79360,"estimated_new_tokens":659,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1823,"cache_hit":0,"new_prefill":80019,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":94269,"ongoing_decode_tokens":94269,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2008,"cache_hit":79360,"new_prefill":659,"score_linear":14909.0,"score_lmetric":659},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":38102,"ongoing_decode_tokens":0,"pending_prefill_tokens":1238,"num_requests":1,"active_p_offloads":0,"cached_blocks":2353,"cache_hit":0,"new_prefill":80019,"score_linear":38102.0,"score_lmetric":81257},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":0,"new_prefill":80019,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":80717,"ongoing_decode_tokens":80717,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2760,"cache_hit":0,"new_prefill":80019,"score_linear":80717.0,"score_lmetric":80019},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":0,"new_prefill":80019,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":80019,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":12288,"new_prefill":67731,"score_linear":-12288.0,"score_lmetric":0}],"chosen_score_linear":14909.0,"chosen_score_lmetric":659,"t_first_token":349697.31719848,"t_first_token_unix":1779780562.331142,"t_done":349698.095312232,"t_done_unix":1779780563.109256},{"request_id":"1253804:21:1640487:1009","session_id":"1253804","input_length":94269,"t_proxy_recv":349686.720712163,"t_decision_unix":1779780551.7346535,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":93696,"affinity_cache_ratio":0.9939216497470006,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":93696,"estimated_new_tokens":573,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1823,"cache_hit":0,"new_prefill":94269,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2008,"cache_hit":93696,"new_prefill":573,"score_linear":-93696.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2353,"cache_hit":0,"new_prefill":94269,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":3584,"new_prefill":90685,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":80717,"ongoing_decode_tokens":80717,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2760,"cache_hit":0,"new_prefill":94269,"score_linear":80717.0,"score_lmetric":94269},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":3584,"new_prefill":90685,"score_linear":-3584.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":94269,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":94269,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-93696.0,"chosen_score_lmetric":0,"t_first_token":349686.921785193,"t_first_token_unix":1779780551.9357283,"t_done":349701.803576666,"t_done_unix":1779780566.8175201},{"request_id":"1399948:44:1644970:1012","session_id":"1399948","input_length":38308,"t_proxy_recv":349701.833471731,"t_decision_unix":1779780566.8474133,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":37888,"affinity_cache_ratio":0.9890362326407017,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":37888,"estimated_new_tokens":420,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1823,"cache_hit":0,"new_prefill":38308,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2011,"cache_hit":0,"new_prefill":38308,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2355,"cache_hit":37888,"new_prefill":420,"score_linear":-37888.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":0,"new_prefill":38308,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2762,"cache_hit":0,"new_prefill":38308,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":0,"new_prefill":38308,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":38308,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":6144,"new_prefill":32164,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-37888.0,"chosen_score_lmetric":0,"t_first_token":349701.94428876,"t_first_token_unix":1779780566.9582322,"t_done":349703.476731897,"t_done_unix":1779780568.4906754},{"request_id":"1253804:22:1650001:1013","session_id":"1253804","input_length":95715,"t_proxy_recv":349718.424634157,"t_decision_unix":1779780583.4385753,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":95232,"affinity_cache_ratio":0.9949537690017238,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":95232,"estimated_new_tokens":483,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1823,"cache_hit":0,"new_prefill":95715,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2011,"cache_hit":95232,"new_prefill":483,"score_linear":-95232.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2356,"cache_hit":0,"new_prefill":95715,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":3584,"new_prefill":92131,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2762,"cache_hit":0,"new_prefill":95715,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":3584,"new_prefill":92131,"score_linear":-3584.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":95715,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":95715,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-95232.0,"chosen_score_lmetric":0,"t_first_token":349718.613921754,"t_first_token_unix":1779780583.627865,"t_done":349719.115216532,"t_done_unix":1779780584.1291597},{"request_id":"1399948:45:1650679:1014","session_id":"1399948","input_length":38758,"t_proxy_recv":349720.874241956,"t_decision_unix":1779780585.8881834,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":38400,"affinity_cache_ratio":0.9907631972754012,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":38400,"estimated_new_tokens":358,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1823,"cache_hit":0,"new_prefill":38758,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2012,"cache_hit":0,"new_prefill":38758,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2356,"cache_hit":38400,"new_prefill":358,"score_linear":-38400.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":0,"new_prefill":38758,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2762,"cache_hit":0,"new_prefill":38758,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":0,"new_prefill":38758,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":38758,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":6144,"new_prefill":32614,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-38400.0,"chosen_score_lmetric":0,"t_first_token":349721.019062115,"t_first_token_unix":1779780586.0330055,"t_done":349721.616682104,"t_done_unix":1779780586.6306252},{"request_id":"1253804:23:1651448:1015","session_id":"1253804","input_length":96944,"t_proxy_recv":349723.174229759,"t_decision_unix":1779780588.1881711,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":95744,"affinity_cache_ratio":0.9876217197557353,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":95744,"estimated_new_tokens":1200,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1823,"cache_hit":0,"new_prefill":96944,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2012,"cache_hit":95744,"new_prefill":1200,"score_linear":-95744.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2356,"cache_hit":0,"new_prefill":96944,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":3584,"new_prefill":93360,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2762,"cache_hit":0,"new_prefill":96944,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":3584,"new_prefill":93360,"score_linear":-3584.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":96944,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":96944,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-95744.0,"chosen_score_lmetric":0,"t_first_token":349724.01293273,"t_first_token_unix":1779780589.0268767,"t_done":349724.424905558,"t_done_unix":1779780589.4388492},{"request_id":"1399948:46:1658183:1018","session_id":"1399948","input_length":39837,"t_proxy_recv":349746.012246885,"t_decision_unix":1779780611.0261881,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":38400,"affinity_cache_ratio":0.9639280066270051,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":38400,"estimated_new_tokens":1437,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1823,"cache_hit":0,"new_prefill":39837,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":80406,"ongoing_decode_tokens":80406,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2014,"cache_hit":0,"new_prefill":39837,"score_linear":80406.0,"score_lmetric":39837},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2356,"cache_hit":38400,"new_prefill":1437,"score_linear":-38400.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":0,"new_prefill":39837,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":83204,"ongoing_decode_tokens":83204,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2762,"cache_hit":0,"new_prefill":39837,"score_linear":83204.0,"score_lmetric":39837},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":0,"new_prefill":39837,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":39837,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":6144,"new_prefill":33693,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-38400.0,"chosen_score_lmetric":0,"t_first_token":349746.395087626,"t_first_token_unix":1779780611.409031,"t_done":349747.258556139,"t_done_unix":1779780612.2724996},{"request_id":"1253804:24:1659020:1019","session_id":"1253804","input_length":97769,"t_proxy_recv":349748.907127096,"t_decision_unix":1779780613.9210682,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":96768,"affinity_cache_ratio":0.9897615808691916,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":96768,"estimated_new_tokens":1001,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1823,"cache_hit":0,"new_prefill":97769,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":80406,"ongoing_decode_tokens":80406,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2014,"cache_hit":96768,"new_prefill":1001,"score_linear":-16362.0,"score_lmetric":1001},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2359,"cache_hit":0,"new_prefill":97769,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":3584,"new_prefill":94185,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":83204,"ongoing_decode_tokens":83204,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2762,"cache_hit":0,"new_prefill":97769,"score_linear":83204.0,"score_lmetric":97769},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":3584,"new_prefill":94185,"score_linear":-3584.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":97769,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":97769,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-16362.0,"chosen_score_lmetric":1001,"t_first_token":349749.571025675,"t_first_token_unix":1779780614.584969,"t_done":349750.204043237,"t_done_unix":1779780615.2179873},{"request_id":"1268861:25:1653400:1016","session_id":"1268861","input_length":80406,"t_proxy_recv":349729.613765781,"t_decision_unix":1779780594.6277072,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":79872,"affinity_cache_ratio":0.9933587045742855,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":79872,"estimated_new_tokens":534,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1823,"cache_hit":0,"new_prefill":80406,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2014,"cache_hit":79872,"new_prefill":534,"score_linear":-79872.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2356,"cache_hit":0,"new_prefill":80406,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":0,"new_prefill":80406,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2762,"cache_hit":0,"new_prefill":80406,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":0,"new_prefill":80406,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":80406,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":12288,"new_prefill":68118,"score_linear":-12288.0,"score_lmetric":0}],"chosen_score_linear":-79872.0,"chosen_score_lmetric":0,"t_first_token":349730.013267709,"t_first_token_unix":1779780595.0272114,"t_done":349755.300862844,"t_done_unix":1779780620.3148062},{"request_id":"1253804:25:1662273:1020","session_id":"1253804","input_length":98187,"t_proxy_recv":349759.770314579,"t_decision_unix":1779780624.7842557,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":97792,"affinity_cache_ratio":0.9959770641734649,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":97792,"estimated_new_tokens":395,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1823,"cache_hit":0,"new_prefill":98187,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2018,"cache_hit":97792,"new_prefill":395,"score_linear":-97792.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2359,"cache_hit":0,"new_prefill":98187,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":3584,"new_prefill":94603,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":83204,"ongoing_decode_tokens":83204,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2762,"cache_hit":0,"new_prefill":98187,"score_linear":83204.0,"score_lmetric":98187},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":3584,"new_prefill":94603,"score_linear":-3584.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":98187,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":98187,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-97792.0,"chosen_score_lmetric":0,"t_first_token":349760.251106127,"t_first_token_unix":1779780625.2650497,"t_done":349760.663174551,"t_done_unix":1779780625.677118},{"request_id":"1294611:15:1657992:1017","session_id":"1294611","input_length":83204,"t_proxy_recv":349745.533220835,"t_decision_unix":1779780610.547162,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":82944,"affinity_cache_ratio":0.9968751502331619,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":82944,"estimated_new_tokens":260,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1823,"cache_hit":0,"new_prefill":83204,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":80406,"ongoing_decode_tokens":80406,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2014,"cache_hit":0,"new_prefill":83204,"score_linear":80406.0,"score_lmetric":83204},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2356,"cache_hit":0,"new_prefill":83204,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":0,"new_prefill":83204,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2762,"cache_hit":82944,"new_prefill":260,"score_linear":-82944.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":12800,"new_prefill":70404,"score_linear":-12800.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":83204,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":12800,"new_prefill":70404,"score_linear":-12800.0,"score_lmetric":0}],"chosen_score_linear":-82944.0,"chosen_score_lmetric":0,"t_first_token":349745.737336693,"t_first_token_unix":1779780610.7512803,"t_done":349767.799063768,"t_done_unix":1779780632.813011},{"request_id":"1399948:47:1664702:1022","session_id":"1399948","input_length":40261,"t_proxy_recv":349768.373069012,"t_decision_unix":1779780633.3870103,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":39936,"affinity_cache_ratio":0.9919276719405876,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":39936,"estimated_new_tokens":325,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1823,"cache_hit":0,"new_prefill":40261,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":98254,"ongoing_decode_tokens":98254,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2018,"cache_hit":0,"new_prefill":40261,"score_linear":98254.0,"score_lmetric":40261},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2359,"cache_hit":39936,"new_prefill":325,"score_linear":-39936.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":0,"new_prefill":40261,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2763,"cache_hit":0,"new_prefill":40261,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":0,"new_prefill":40261,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":40261,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":6144,"new_prefill":34117,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-39936.0,"chosen_score_lmetric":0,"t_first_token":349768.592951544,"t_first_token_unix":1779780633.606896,"t_done":349769.019591464,"t_done_unix":1779780634.0335355},{"request_id":"1356885:8:1665690:1023","session_id":"1356885","input_length":132537,"t_proxy_recv":349771.588128846,"t_decision_unix":1779780636.6020703,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":131584,"affinity_cache_ratio":0.9928095550676415,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":131584,"estimated_new_tokens":953,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1823,"cache_hit":131584,"new_prefill":953,"score_linear":-131584.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":98254,"ongoing_decode_tokens":98254,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2018,"cache_hit":1024,"new_prefill":131513,"score_linear":97230.0,"score_lmetric":131513},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2359,"cache_hit":0,"new_prefill":132537,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":0,"new_prefill":132537,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2763,"cache_hit":6656,"new_prefill":125881,"score_linear":-6656.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":0,"new_prefill":132537,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":132537,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":132537,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-131584.0,"chosen_score_lmetric":0,"t_first_token":349772.264887557,"t_first_token_unix":1779780637.2788317,"t_done":349774.036519588,"t_done_unix":1779780639.0504632},{"request_id":"1399948:48:1666394:1024","session_id":"1399948","input_length":40343,"t_proxy_recv":349774.002539974,"t_decision_unix":1779780639.0164812,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":39936,"affinity_cache_ratio":0.9899115088119377,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":39936,"estimated_new_tokens":407,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":132537,"ongoing_decode_tokens":132537,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1823,"cache_hit":0,"new_prefill":40343,"score_linear":132537.0,"score_lmetric":40343},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":98254,"ongoing_decode_tokens":98254,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2018,"cache_hit":0,"new_prefill":40343,"score_linear":98254.0,"score_lmetric":40343},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2359,"cache_hit":39936,"new_prefill":407,"score_linear":-39936.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":0,"new_prefill":40343,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2763,"cache_hit":0,"new_prefill":40343,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":0,"new_prefill":40343,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":40343,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":6144,"new_prefill":34199,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-39936.0,"chosen_score_lmetric":0,"t_first_token":349774.072959347,"t_first_token_unix":1779780639.0869024,"t_done":349775.472433741,"t_done_unix":1779780640.4863768},{"request_id":"1399948:49:1668825:1025","session_id":"1399948","input_length":40642,"t_proxy_recv":349782.566270864,"t_decision_unix":1779780647.5802119,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":40448,"affinity_cache_ratio":0.9952266128635402,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":40448,"estimated_new_tokens":194,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1825,"cache_hit":0,"new_prefill":40642,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":98254,"ongoing_decode_tokens":98254,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2018,"cache_hit":0,"new_prefill":40642,"score_linear":98254.0,"score_lmetric":40642},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2360,"cache_hit":40448,"new_prefill":194,"score_linear":-40448.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":0,"new_prefill":40642,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2763,"cache_hit":0,"new_prefill":40642,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":0,"new_prefill":40642,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":40642,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":6144,"new_prefill":34498,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-40448.0,"chosen_score_lmetric":0,"t_first_token":349782.685643983,"t_first_token_unix":1779780647.6995873,"t_done":349783.771971895,"t_done_unix":1779780648.7859151},{"request_id":"1294611:15:1673197:1026","session_id":"1294611","input_length":85542,"t_proxy_recv":349797.781842374,"t_decision_unix":1779780662.7957828,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":84992,"affinity_cache_ratio":0.9935704098571462,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":84992,"estimated_new_tokens":550,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1825,"cache_hit":0,"new_prefill":85542,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":98254,"ongoing_decode_tokens":98254,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2018,"cache_hit":0,"new_prefill":85542,"score_linear":98254.0,"score_lmetric":85542},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2360,"cache_hit":0,"new_prefill":85542,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":0,"new_prefill":85542,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2763,"cache_hit":84992,"new_prefill":550,"score_linear":-84992.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":12800,"new_prefill":72742,"score_linear":-12800.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":85542,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":12800,"new_prefill":72742,"score_linear":-12800.0,"score_lmetric":0}],"chosen_score_linear":-84992.0,"chosen_score_lmetric":0,"t_first_token":349797.965777059,"t_first_token_unix":1779780662.9797204,"t_done":349799.568171585,"t_done_unix":1779780664.5821154},{"request_id":"1253804:26:1663639:1021","session_id":"1253804","input_length":98254,"t_proxy_recv":349764.6035731,"t_decision_unix":1779780629.6175141,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":97792,"affinity_cache_ratio":0.9952979013577056,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":97792,"estimated_new_tokens":462,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1823,"cache_hit":0,"new_prefill":98254,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2018,"cache_hit":97792,"new_prefill":462,"score_linear":-97792.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2359,"cache_hit":0,"new_prefill":98254,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":3584,"new_prefill":94670,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":83204,"ongoing_decode_tokens":83204,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2762,"cache_hit":0,"new_prefill":98254,"score_linear":83204.0,"score_lmetric":98254},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":3584,"new_prefill":94670,"score_linear":-3584.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":98254,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":98254,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-97792.0,"chosen_score_lmetric":0,"t_first_token":349764.760666112,"t_first_token_unix":1779780629.7746093,"t_done":349801.189546785,"t_done_unix":1779780666.2034905},{"request_id":"1294611:15:1676489:1027","session_id":"1294611","input_length":85722,"t_proxy_recv":349809.170011182,"t_decision_unix":1779780674.1839523,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":85504,"affinity_cache_ratio":0.9974568955460675,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":85504,"estimated_new_tokens":218,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1825,"cache_hit":0,"new_prefill":85722,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2019,"cache_hit":0,"new_prefill":85722,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2360,"cache_hit":0,"new_prefill":85722,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":0,"new_prefill":85722,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2764,"cache_hit":85504,"new_prefill":218,"score_linear":-85504.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":12800,"new_prefill":72922,"score_linear":-12800.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":85722,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":12800,"new_prefill":72922,"score_linear":-12800.0,"score_lmetric":0}],"chosen_score_linear":-85504.0,"chosen_score_lmetric":0,"t_first_token":349809.290994268,"t_first_token_unix":1779780674.304938,"t_done":349811.885956484,"t_done_unix":1779780676.8999002},{"request_id":"1399948:50:1677482:1028","session_id":"1399948","input_length":42648,"t_proxy_recv":349812.42018334,"t_decision_unix":1779780677.4341247,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":40448,"affinity_cache_ratio":0.9484149315325455,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":40448,"estimated_new_tokens":2200,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1825,"cache_hit":0,"new_prefill":42648,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2019,"cache_hit":0,"new_prefill":42648,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2360,"cache_hit":40448,"new_prefill":2200,"score_linear":-40448.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":0,"new_prefill":42648,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2764,"cache_hit":0,"new_prefill":42648,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":0,"new_prefill":42648,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":42648,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":6144,"new_prefill":36504,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-40448.0,"chosen_score_lmetric":0,"t_first_token":349813.020560747,"t_first_token_unix":1779780678.0345047,"t_done":349816.68564097,"t_done_unix":1779780681.6995845},{"request_id":"1294611:15:1679070:1029","session_id":"1294611","input_length":86602,"t_proxy_recv":349817.850103431,"t_decision_unix":1779780682.8640444,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":85504,"affinity_cache_ratio":0.987321308976698,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":85504,"estimated_new_tokens":1098,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1825,"cache_hit":0,"new_prefill":86602,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2019,"cache_hit":0,"new_prefill":86602,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2365,"cache_hit":0,"new_prefill":86602,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":0,"new_prefill":86602,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2764,"cache_hit":85504,"new_prefill":1098,"score_linear":-85504.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":12800,"new_prefill":73802,"score_linear":-12800.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":86602,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":12800,"new_prefill":73802,"score_linear":-12800.0,"score_lmetric":0}],"chosen_score_linear":-85504.0,"chosen_score_lmetric":0,"t_first_token":349818.286613123,"t_first_token_unix":1779780683.300557,"t_done":349821.07350979,"t_done_unix":1779780686.0874534},{"request_id":"1253804:27:1683133:1031","session_id":"1253804","input_length":101743,"t_proxy_recv":349831.47679848,"t_decision_unix":1779780696.4907398,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":101376,"affinity_cache_ratio":0.9963928722369106,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":101376,"estimated_new_tokens":367,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1825,"cache_hit":0,"new_prefill":101743,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2019,"cache_hit":101376,"new_prefill":367,"score_linear":-101376.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2365,"cache_hit":0,"new_prefill":101743,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":3584,"new_prefill":98159,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":86925,"ongoing_decode_tokens":86925,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2766,"cache_hit":0,"new_prefill":101743,"score_linear":86925.0,"score_lmetric":101743},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":3584,"new_prefill":98159,"score_linear":-3584.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":101743,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":101743,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-101376.0,"chosen_score_lmetric":0,"t_first_token":349831.672473837,"t_first_token_unix":1779780696.6864169,"t_done":349831.9875604,"t_done_unix":1779780697.0015044},{"request_id":"1294611:15:1682649:1030","session_id":"1294611","input_length":86925,"t_proxy_recv":349829.757723034,"t_decision_unix":1779780694.7716644,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":4,"chosen_idx":4,"affinity_cache_hit":86528,"affinity_cache_ratio":0.9954328444060973,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":86528,"estimated_new_tokens":397,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8004","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1825,"cache_hit":0,"new_prefill":86925,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2019,"cache_hit":0,"new_prefill":86925,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2365,"cache_hit":0,"new_prefill":86925,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":0,"new_prefill":86925,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2766,"cache_hit":86528,"new_prefill":397,"score_linear":-86528.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":12800,"new_prefill":74125,"score_linear":-12800.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":86925,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":12800,"new_prefill":74125,"score_linear":-12800.0,"score_lmetric":0}],"chosen_score_linear":-86528.0,"chosen_score_lmetric":0,"t_first_token":349829.896114514,"t_first_token_unix":1779780694.910058,"t_done":349833.879045174,"t_done_unix":1779780698.892989},{"request_id":"1253804:28:1685257:1032","session_id":"1253804","input_length":101794,"t_proxy_recv":349838.477947426,"t_decision_unix":1779780703.4918883,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":101376,"affinity_cache_ratio":0.9958936676031986,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":101376,"estimated_new_tokens":418,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1825,"cache_hit":0,"new_prefill":101794,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2019,"cache_hit":101376,"new_prefill":418,"score_linear":-101376.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2365,"cache_hit":0,"new_prefill":101794,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":3584,"new_prefill":98210,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":101794,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":3584,"new_prefill":98210,"score_linear":-3584.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":101794,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":101794,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-101376.0,"chosen_score_lmetric":0,"t_first_token":349838.614656187,"t_first_token_unix":1779780703.6286004,"t_done":349839.340146932,"t_done_unix":1779780704.3540902},{"request_id":"1399948:51:1686626:1033","session_id":"1399948","input_length":43407,"t_proxy_recv":349843.409276962,"t_decision_unix":1779780708.4232185,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":43008,"affinity_cache_ratio":0.9908079342041606,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":43008,"estimated_new_tokens":399,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1825,"cache_hit":0,"new_prefill":43407,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2019,"cache_hit":0,"new_prefill":43407,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2365,"cache_hit":43008,"new_prefill":399,"score_linear":-43008.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":0,"new_prefill":43407,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":43407,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":0,"new_prefill":43407,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":43407,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":6144,"new_prefill":37263,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-43008.0,"chosen_score_lmetric":0,"t_first_token":349843.567295196,"t_first_token_unix":1779780708.5812385,"t_done":349845.019579891,"t_done_unix":1779780710.0335236},{"request_id":"1253804:29:1688398:1034","session_id":"1253804","input_length":102067,"t_proxy_recv":349849.467939096,"t_decision_unix":1779780714.4818804,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":101376,"affinity_cache_ratio":0.9932299371981149,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":101376,"estimated_new_tokens":691,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1825,"cache_hit":0,"new_prefill":102067,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2019,"cache_hit":101376,"new_prefill":691,"score_linear":-101376.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2366,"cache_hit":0,"new_prefill":102067,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":3584,"new_prefill":98483,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":102067,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":3584,"new_prefill":98483,"score_linear":-3584.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":102067,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":102067,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-101376.0,"chosen_score_lmetric":0,"t_first_token":349849.776586032,"t_first_token_unix":1779780714.7905295,"t_done":349850.803236545,"t_done_unix":1779780715.8171797},{"request_id":"1399948:52:1689144:1035","session_id":"1399948","input_length":43673,"t_proxy_recv":349852.044963501,"t_decision_unix":1779780717.058905,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":43520,"affinity_cache_ratio":0.9964966913195796,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":43520,"estimated_new_tokens":153,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1825,"cache_hit":0,"new_prefill":43673,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2020,"cache_hit":0,"new_prefill":43673,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2366,"cache_hit":43520,"new_prefill":153,"score_linear":-43520.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":0,"new_prefill":43673,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":43673,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":0,"new_prefill":43673,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":43673,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":6144,"new_prefill":37529,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-43520.0,"chosen_score_lmetric":0,"t_first_token":349852.157634813,"t_first_token_unix":1779780717.171578,"t_done":349852.90122549,"t_done_unix":1779780717.9151688},{"request_id":"1356885:9:1689871:1036","session_id":"1356885","input_length":132823,"t_proxy_recv":349854.424225105,"t_decision_unix":1779780719.4381664,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":132608,"affinity_cache_ratio":0.9983813044427546,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":132608,"estimated_new_tokens":215,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1825,"cache_hit":132608,"new_prefill":215,"score_linear":-132608.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2020,"cache_hit":1024,"new_prefill":131799,"score_linear":-1024.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2366,"cache_hit":0,"new_prefill":132823,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":0,"new_prefill":132823,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":6656,"new_prefill":126167,"score_linear":-6656.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":0,"new_prefill":132823,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":132823,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":132823,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-132608.0,"chosen_score_lmetric":0,"t_first_token":349854.70968359,"t_first_token_unix":1779780719.7236273,"t_done":349856.406022217,"t_done_unix":1779780721.4199657},{"request_id":"1253804:30:1690362:1037","session_id":"1253804","input_length":102214,"t_proxy_recv":349856.078022574,"t_decision_unix":1779780721.0919635,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":101888,"affinity_cache_ratio":0.9968106130275696,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":101888,"estimated_new_tokens":326,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":132823,"ongoing_decode_tokens":132823,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1825,"cache_hit":0,"new_prefill":102214,"score_linear":132823.0,"score_lmetric":102214},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2020,"cache_hit":101888,"new_prefill":326,"score_linear":-101888.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2366,"cache_hit":0,"new_prefill":102214,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":3584,"new_prefill":98630,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":102214,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":3584,"new_prefill":98630,"score_linear":-3584.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":102214,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":102214,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-101888.0,"chosen_score_lmetric":0,"t_first_token":349856.291983506,"t_first_token_unix":1779780721.3059273,"t_done":349857.039042504,"t_done_unix":1779780722.0529866},{"request_id":"1253804:31:1691981:1038","session_id":"1253804","input_length":102322,"t_proxy_recv":349861.650373489,"t_decision_unix":1779780726.664315,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":101888,"affinity_cache_ratio":0.9957584879107132,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":101888,"estimated_new_tokens":434,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1825,"cache_hit":0,"new_prefill":102322,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2020,"cache_hit":101888,"new_prefill":434,"score_linear":-101888.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2366,"cache_hit":0,"new_prefill":102322,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":3584,"new_prefill":98738,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":102322,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":3584,"new_prefill":98738,"score_linear":-3584.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":102322,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":102322,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-101888.0,"chosen_score_lmetric":0,"t_first_token":349861.83072162,"t_first_token_unix":1779780726.8446648,"t_done":349862.195817398,"t_done_unix":1779780727.2097611},{"request_id":"1399948:53:1692115:1039","session_id":"1399948","input_length":43938,"t_proxy_recv":349862.16694081,"t_decision_unix":1779780727.1808825,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":43520,"affinity_cache_ratio":0.9904865947471437,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":43520,"estimated_new_tokens":418,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1825,"cache_hit":0,"new_prefill":43938,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":102322,"ongoing_decode_tokens":102322,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2020,"cache_hit":0,"new_prefill":43938,"score_linear":102322.0,"score_lmetric":43938},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2366,"cache_hit":43520,"new_prefill":418,"score_linear":-43520.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":0,"new_prefill":43938,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":43938,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":0,"new_prefill":43938,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":43938,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":6144,"new_prefill":37794,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-43520.0,"chosen_score_lmetric":0,"t_first_token":349862.317988371,"t_first_token_unix":1779780727.3319318,"t_done":349863.700419849,"t_done_unix":1779780728.7143633},{"request_id":"1253804:32:1693715:1040","session_id":"1253804","input_length":102509,"t_proxy_recv":349867.738907807,"t_decision_unix":1779780732.7528489,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":101888,"affinity_cache_ratio":0.9939419953369948,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":101888,"estimated_new_tokens":621,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1825,"cache_hit":0,"new_prefill":102509,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2020,"cache_hit":101888,"new_prefill":621,"score_linear":-101888.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2367,"cache_hit":0,"new_prefill":102509,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":3584,"new_prefill":98925,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":102509,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":3584,"new_prefill":98925,"score_linear":-3584.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":102509,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":102509,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-101888.0,"chosen_score_lmetric":0,"t_first_token":349868.045828031,"t_first_token_unix":1779780733.0597715,"t_done":349868.832142711,"t_done_unix":1779780733.8460865},{"request_id":"1399948:54:1694570:1041","session_id":"1399948","input_length":44182,"t_proxy_recv":349870.82759853,"t_decision_unix":1779780735.8415399,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":44032,"affinity_cache_ratio":0.9966049522429948,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":44032,"estimated_new_tokens":150,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1825,"cache_hit":0,"new_prefill":44182,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2021,"cache_hit":0,"new_prefill":44182,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2367,"cache_hit":44032,"new_prefill":150,"score_linear":-44032.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":0,"new_prefill":44182,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":44182,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":0,"new_prefill":44182,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":44182,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":6144,"new_prefill":38038,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-44032.0,"chosen_score_lmetric":0,"t_first_token":349870.933702578,"t_first_token_unix":1779780735.947646,"t_done":349871.216267031,"t_done_unix":1779780736.2302108},{"request_id":"1253804:33:1695393:1042","session_id":"1253804","input_length":102632,"t_proxy_recv":349873.49094023,"t_decision_unix":1779780738.5048816,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":102400,"affinity_cache_ratio":0.9977394964533479,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":102400,"estimated_new_tokens":232,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1825,"cache_hit":0,"new_prefill":102632,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2021,"cache_hit":102400,"new_prefill":232,"score_linear":-102400.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2367,"cache_hit":0,"new_prefill":102632,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":3584,"new_prefill":99048,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":102632,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":3584,"new_prefill":99048,"score_linear":-3584.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":102632,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":102632,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-102400.0,"chosen_score_lmetric":0,"t_first_token":349873.705681,"t_first_token_unix":1779780738.7196243,"t_done":349874.189471662,"t_done_unix":1779780739.2034156},{"request_id":"1253804:34:1697569:1043","session_id":"1253804","input_length":102792,"t_proxy_recv":349881.329228799,"t_decision_unix":1779780746.3431702,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":102400,"affinity_cache_ratio":0.9961864736555374,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":102400,"estimated_new_tokens":392,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1825,"cache_hit":0,"new_prefill":102792,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2021,"cache_hit":102400,"new_prefill":392,"score_linear":-102400.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2367,"cache_hit":0,"new_prefill":102792,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":3584,"new_prefill":99208,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":102792,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":3584,"new_prefill":99208,"score_linear":-3584.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":102792,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":102792,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-102400.0,"chosen_score_lmetric":0,"t_first_token":349881.573156884,"t_first_token_unix":1779780746.5871005,"t_done":349884.98825531,"t_done_unix":1779780750.0021985},{"request_id":"1253804:35:1701250:1044","session_id":"1253804","input_length":103349,"t_proxy_recv":349893.866306076,"t_decision_unix":1779780758.8802476,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":102912,"affinity_cache_ratio":0.9957716088205981,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":102912,"estimated_new_tokens":437,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1825,"cache_hit":0,"new_prefill":103349,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2022,"cache_hit":102912,"new_prefill":437,"score_linear":-102912.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2367,"cache_hit":0,"new_prefill":103349,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":3584,"new_prefill":99765,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":103349,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":3584,"new_prefill":99765,"score_linear":-3584.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":103349,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":103349,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-102912.0,"chosen_score_lmetric":0,"t_first_token":349894.18304134,"t_first_token_unix":1779780759.1969848,"t_done":349894.494966398,"t_done_unix":1779780759.5089102},{"request_id":"1253804:36:1702697:1045","session_id":"1253804","input_length":103400,"t_proxy_recv":349899.019997481,"t_decision_unix":1779780764.0339386,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":102912,"affinity_cache_ratio":0.9952804642166344,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":102912,"estimated_new_tokens":488,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1825,"cache_hit":0,"new_prefill":103400,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2022,"cache_hit":102912,"new_prefill":488,"score_linear":-102912.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2367,"cache_hit":0,"new_prefill":103400,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":3584,"new_prefill":99816,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":103400,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":3584,"new_prefill":99816,"score_linear":-3584.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":103400,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":103400,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-102912.0,"chosen_score_lmetric":0,"t_first_token":349899.191483589,"t_first_token_unix":1779780764.205427,"t_done":349899.619178127,"t_done_unix":1779780764.633121},{"request_id":"1399948:55:1704272:1047","session_id":"1399948","input_length":45284,"t_proxy_recv":349904.289813205,"t_decision_unix":1779780769.3037546,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":44032,"affinity_cache_ratio":0.9723522657009098,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":44032,"estimated_new_tokens":1252,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1825,"cache_hit":0,"new_prefill":45284,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":103471,"ongoing_decode_tokens":103471,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2023,"cache_hit":0,"new_prefill":45284,"score_linear":103471.0,"score_lmetric":45284},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2367,"cache_hit":44032,"new_prefill":1252,"score_linear":-44032.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":0,"new_prefill":45284,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":45284,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":0,"new_prefill":45284,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":45284,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":6144,"new_prefill":39140,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-44032.0,"chosen_score_lmetric":0,"t_first_token":349904.724441396,"t_first_token_unix":1779780769.738385,"t_done":349906.412667208,"t_done_unix":1779780771.426611},{"request_id":"1356885:10:1706624:1048","session_id":"1356885","input_length":133014,"t_proxy_recv":349912.617596635,"t_decision_unix":1779780777.6315382,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":132608,"affinity_cache_ratio":0.9969476897168719,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":132608,"estimated_new_tokens":406,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1825,"cache_hit":132608,"new_prefill":406,"score_linear":-132608.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":103471,"ongoing_decode_tokens":103471,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2023,"cache_hit":1024,"new_prefill":131990,"score_linear":102447.0,"score_lmetric":131990},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2369,"cache_hit":0,"new_prefill":133014,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":0,"new_prefill":133014,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":6656,"new_prefill":126358,"score_linear":-6656.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":0,"new_prefill":133014,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":133014,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":133014,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-132608.0,"chosen_score_lmetric":0,"t_first_token":349912.870024034,"t_first_token_unix":1779780777.8839674,"t_done":349914.345067628,"t_done_unix":1779780779.3590114},{"request_id":"1253804:37:1704071:1046","session_id":"1253804","input_length":103471,"t_proxy_recv":349903.493737123,"t_decision_unix":1779780768.5076785,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":103424,"affinity_cache_ratio":0.9995457664466372,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":103424,"estimated_new_tokens":47,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1825,"cache_hit":0,"new_prefill":103471,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2023,"cache_hit":103424,"new_prefill":47,"score_linear":-103424.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2367,"cache_hit":0,"new_prefill":103471,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":3584,"new_prefill":99887,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":103471,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":3584,"new_prefill":99887,"score_linear":-3584.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":103471,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":103471,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-103424.0,"chosen_score_lmetric":0,"t_first_token":349903.632101999,"t_first_token_unix":1779780768.6460452,"t_done":349919.162451348,"t_done_unix":1779780784.1763945},{"request_id":"1399948:56:1708284:1049","session_id":"1399948","input_length":45571,"t_proxy_recv":349918.327388868,"t_decision_unix":1779780783.3413305,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":45056,"affinity_cache_ratio":0.9886989532816923,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":45056,"estimated_new_tokens":515,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1826,"cache_hit":0,"new_prefill":45571,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":103471,"ongoing_decode_tokens":103471,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2023,"cache_hit":0,"new_prefill":45571,"score_linear":103471.0,"score_lmetric":45571},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2369,"cache_hit":45056,"new_prefill":515,"score_linear":-45056.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":0,"new_prefill":45571,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":45571,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":0,"new_prefill":45571,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":45571,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":6144,"new_prefill":39427,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-45056.0,"chosen_score_lmetric":0,"t_first_token":349918.438413267,"t_first_token_unix":1779780783.4523566,"t_done":349919.341747094,"t_done_unix":1779780784.35569},{"request_id":"1399948:57:1711464:1050","session_id":"1399948","input_length":45756,"t_proxy_recv":349929.118940484,"t_decision_unix":1779780794.1328819,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":45568,"affinity_cache_ratio":0.995891249235073,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":45568,"estimated_new_tokens":188,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1826,"cache_hit":0,"new_prefill":45756,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2025,"cache_hit":0,"new_prefill":45756,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2370,"cache_hit":45568,"new_prefill":188,"score_linear":-45568.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":0,"new_prefill":45756,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":45756,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":0,"new_prefill":45756,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":45756,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":6144,"new_prefill":39612,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-45568.0,"chosen_score_lmetric":0,"t_first_token":349929.230068855,"t_first_token_unix":1779780794.244012,"t_done":349931.725861881,"t_done_unix":1779780796.7398052},{"request_id":"1253804:38:1712824:1051","session_id":"1253804","input_length":104953,"t_proxy_recv":349933.947912117,"t_decision_unix":1779780798.9618535,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":104448,"affinity_cache_ratio":0.9951883223919278,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":104448,"estimated_new_tokens":505,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1826,"cache_hit":0,"new_prefill":104953,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2025,"cache_hit":104448,"new_prefill":505,"score_linear":-104448.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2371,"cache_hit":0,"new_prefill":104953,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":3584,"new_prefill":101369,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":104953,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":3584,"new_prefill":101369,"score_linear":-3584.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":104953,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":104953,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-104448.0,"chosen_score_lmetric":0,"t_first_token":349934.151313922,"t_first_token_unix":1779780799.1652572,"t_done":349934.468420499,"t_done_unix":1779780799.4823642},{"request_id":"1253804:39:1714767:1052","session_id":"1253804","input_length":105004,"t_proxy_recv":349940.534915497,"t_decision_unix":1779780805.5488565,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":104960,"affinity_cache_ratio":0.9995809683440631,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":104960,"estimated_new_tokens":44,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1826,"cache_hit":0,"new_prefill":105004,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2026,"cache_hit":104960,"new_prefill":44,"score_linear":-104960.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2371,"cache_hit":0,"new_prefill":105004,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":3584,"new_prefill":101420,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":105004,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":3584,"new_prefill":101420,"score_linear":-3584.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":105004,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":105004,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-104960.0,"chosen_score_lmetric":0,"t_first_token":349940.677683317,"t_first_token_unix":1779780805.6916268,"t_done":349941.257814047,"t_done_unix":1779780806.2717576},{"request_id":"1253804:40:1716790:1053","session_id":"1253804","input_length":105156,"t_proxy_recv":349947.243491271,"t_decision_unix":1779780812.2574325,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":104960,"affinity_cache_ratio":0.9981361025523984,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":104960,"estimated_new_tokens":196,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1826,"cache_hit":0,"new_prefill":105156,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2026,"cache_hit":104960,"new_prefill":196,"score_linear":-104960.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2371,"cache_hit":0,"new_prefill":105156,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":3584,"new_prefill":101572,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":105156,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":3584,"new_prefill":101572,"score_linear":-3584.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":105156,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":105156,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-104960.0,"chosen_score_lmetric":0,"t_first_token":349947.511436889,"t_first_token_unix":1779780812.5253801,"t_done":349952.170832967,"t_done_unix":1779780817.1847765},{"request_id":"1305906:4:1718131:1054","session_id":"1305906","input_length":47861,"t_proxy_recv":349951.695687571,"t_decision_unix":1779780816.7096288,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":45056,"affinity_cache_ratio":0.9413927832682142,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":45056,"estimated_new_tokens":2805,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1826,"cache_hit":0,"new_prefill":47861,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":105156,"ongoing_decode_tokens":105156,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2026,"cache_hit":0,"new_prefill":47861,"score_linear":105156.0,"score_lmetric":47861},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2371,"cache_hit":45056,"new_prefill":2805,"score_linear":-45056.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":0,"new_prefill":47861,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":47861,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":0,"new_prefill":47861,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":47861,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":47861,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-45056.0,"chosen_score_lmetric":0,"t_first_token":349952.616425757,"t_first_token_unix":1779780817.6303694,"t_done":349958.58940846,"t_done_unix":1779780823.6033518},{"request_id":"1305906:5:1514292:934","session_id":"1305906","input_length":49817,"t_proxy_recv":349958.605526883,"t_decision_unix":1779780823.6194685,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":48128,"affinity_cache_ratio":0.9660959110343859,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":48128,"estimated_new_tokens":1689,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1826,"cache_hit":0,"new_prefill":49817,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2027,"cache_hit":0,"new_prefill":49817,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":37173,"ongoing_decode_tokens":37173,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2377,"cache_hit":48128,"new_prefill":1689,"score_linear":-10955.0,"score_lmetric":1689},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":0,"new_prefill":49817,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":49817,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":0,"new_prefill":49817,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":49817,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":49817,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-10955.0,"chosen_score_lmetric":1689,"t_first_token":349959.265038993,"t_first_token_unix":1779780824.278983,"t_done":349960.182966135,"t_done_unix":1779780825.19691},{"request_id":"1340290:13:1718889:1055","session_id":"1340290","input_length":37173,"t_proxy_recv":349954.324103346,"t_decision_unix":1779780819.3380444,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":35328,"affinity_cache_ratio":0.9503672020014526,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":35328,"estimated_new_tokens":1845,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1826,"cache_hit":0,"new_prefill":37173,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2027,"cache_hit":0,"new_prefill":37173,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":47861,"ongoing_decode_tokens":47861,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2371,"cache_hit":35328,"new_prefill":1845,"score_linear":12533.0,"score_lmetric":1845},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":0,"new_prefill":37173,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":37173,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":0,"new_prefill":37173,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":36149,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":37173,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":12533.0,"chosen_score_lmetric":1845,"t_first_token":349958.151892581,"t_first_token_unix":1779780823.1658363,"t_done":349960.825056166,"t_done_unix":1779780825.8389995},{"request_id":"1253804:41:1721979:1057","session_id":"1253804","input_length":105638,"t_proxy_recv":349964.920551899,"t_decision_unix":1779780829.934493,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":105472,"affinity_cache_ratio":0.9984285957704614,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":105472,"estimated_new_tokens":166,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1826,"cache_hit":0,"new_prefill":105638,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2027,"cache_hit":105472,"new_prefill":166,"score_linear":-105472.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":37438,"ongoing_decode_tokens":37438,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2384,"cache_hit":0,"new_prefill":105638,"score_linear":37438.0,"score_lmetric":105638},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":3584,"new_prefill":102054,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":105638,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":3584,"new_prefill":102054,"score_linear":-3584.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":105638,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":105638,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-105472.0,"chosen_score_lmetric":0,"t_first_token":349965.116654125,"t_first_token_unix":1779780830.1305974,"t_done":349965.772812634,"t_done_unix":1779780830.7867563},{"request_id":"1340290:14:1721731:1056","session_id":"1340290","input_length":37438,"t_proxy_recv":349964.047073839,"t_decision_unix":1779780829.0610151,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":37376,"affinity_cache_ratio":0.998343928628666,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":37376,"estimated_new_tokens":62,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1826,"cache_hit":0,"new_prefill":37438,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2027,"cache_hit":0,"new_prefill":37438,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2384,"cache_hit":37376,"new_prefill":62,"score_linear":-37376.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":0,"new_prefill":37438,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":37438,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":0,"new_prefill":37438,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":36414,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":37438,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-37376.0,"chosen_score_lmetric":0,"t_first_token":349964.125723005,"t_first_token_unix":1779780829.139666,"t_done":349965.911145637,"t_done_unix":1779780830.9250891},{"request_id":"1253804:42:1724453:1058","session_id":"1253804","input_length":105715,"t_proxy_recv":349973.422319233,"t_decision_unix":1779780838.4362607,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":105472,"affinity_cache_ratio":0.9977013668826562,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":105472,"estimated_new_tokens":243,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1826,"cache_hit":0,"new_prefill":105715,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2027,"cache_hit":105472,"new_prefill":243,"score_linear":-105472.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2384,"cache_hit":0,"new_prefill":105715,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":3584,"new_prefill":102131,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":105715,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":3584,"new_prefill":102131,"score_linear":-3584.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":105715,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":105715,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-105472.0,"chosen_score_lmetric":0,"t_first_token":349973.574809992,"t_first_token_unix":1779780838.5887535,"t_done":349974.35986027,"t_done_unix":1779780839.3738036},{"request_id":"1253804:43:1726322:1060","session_id":"1253804","input_length":105845,"t_proxy_recv":349979.494521534,"t_decision_unix":1779780844.508463,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":105472,"affinity_cache_ratio":0.9964759790259341,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":105472,"estimated_new_tokens":373,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1826,"cache_hit":0,"new_prefill":105845,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2027,"cache_hit":105472,"new_prefill":373,"score_linear":-105472.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":37724,"ongoing_decode_tokens":37724,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2384,"cache_hit":0,"new_prefill":105845,"score_linear":37724.0,"score_lmetric":105845},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":3584,"new_prefill":102261,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":105845,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":3584,"new_prefill":102261,"score_linear":-3584.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":105845,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":105845,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-105472.0,"chosen_score_lmetric":0,"t_first_token":349979.690578503,"t_first_token_unix":1779780844.7045217,"t_done":349980.388293657,"t_done_unix":1779780845.4022372},{"request_id":"1399948:58:1726693:1061","session_id":"1399948","input_length":46886,"t_proxy_recv":349980.713269664,"t_decision_unix":1779780845.727211,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":46080,"affinity_cache_ratio":0.982809367401783,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":46080,"estimated_new_tokens":806,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1826,"cache_hit":0,"new_prefill":46886,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2027,"cache_hit":0,"new_prefill":46886,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":37724,"ongoing_decode_tokens":37724,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2384,"cache_hit":46080,"new_prefill":806,"score_linear":-8356.0,"score_lmetric":806},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":0,"new_prefill":46886,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":46886,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":0,"new_prefill":46886,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":46886,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":6144,"new_prefill":40742,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-8356.0,"chosen_score_lmetric":806,"t_first_token":349981.068789353,"t_first_token_unix":1779780846.0827324,"t_done":349981.260791917,"t_done_unix":1779780846.2747347},{"request_id":"1399948:59:1728305:1062","session_id":"1399948","input_length":46927,"t_proxy_recv":349986.004655057,"t_decision_unix":1779780851.0185964,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":46592,"affinity_cache_ratio":0.9928612525838004,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":46592,"estimated_new_tokens":335,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1826,"cache_hit":0,"new_prefill":46927,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2027,"cache_hit":0,"new_prefill":46927,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":37724,"ongoing_decode_tokens":37724,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2385,"cache_hit":46592,"new_prefill":335,"score_linear":-8868.0,"score_lmetric":335},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":0,"new_prefill":46927,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":46927,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":0,"new_prefill":46927,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":46927,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":6144,"new_prefill":40783,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-8868.0,"chosen_score_lmetric":335,"t_first_token":349986.093365127,"t_first_token_unix":1779780851.1073084,"t_done":349986.30812381,"t_done_unix":1779780851.322067},{"request_id":"1253804:44:1728625:1063","session_id":"1253804","input_length":106070,"t_proxy_recv":349987.196266648,"t_decision_unix":1779780852.210208,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":105472,"affinity_cache_ratio":0.9943622136325069,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":105472,"estimated_new_tokens":598,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1826,"cache_hit":0,"new_prefill":106070,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2027,"cache_hit":105472,"new_prefill":598,"score_linear":-105472.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":37724,"ongoing_decode_tokens":37724,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2385,"cache_hit":0,"new_prefill":106070,"score_linear":37724.0,"score_lmetric":106070},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":3584,"new_prefill":102486,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":106070,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":3584,"new_prefill":102486,"score_linear":-3584.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":106070,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":106070,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-105472.0,"chosen_score_lmetric":0,"t_first_token":349987.508827381,"t_first_token_unix":1779780852.5227704,"t_done":349988.560218123,"t_done_unix":1779780853.5741618},{"request_id":"1340290:15:1725612:1059","session_id":"1340290","input_length":37724,"t_proxy_recv":349977.229139758,"t_decision_unix":1779780842.2430813,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":37376,"affinity_cache_ratio":0.9907751033824621,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":37376,"estimated_new_tokens":348,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1826,"cache_hit":0,"new_prefill":37724,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2027,"cache_hit":0,"new_prefill":37724,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2384,"cache_hit":37376,"new_prefill":348,"score_linear":-37376.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":0,"new_prefill":37724,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":37724,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":0,"new_prefill":37724,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":36700,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":37724,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-37376.0,"chosen_score_lmetric":0,"t_first_token":349977.302710491,"t_first_token_unix":1779780842.3166535,"t_done":349995.427108065,"t_done_unix":1779780860.4410515},{"request_id":"1253804:45:1732408:1064","session_id":"1253804","input_length":106394,"t_proxy_recv":350000.164359338,"t_decision_unix":1779780865.1783004,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":105984,"affinity_cache_ratio":0.9961463992330395,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":105984,"estimated_new_tokens":410,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1826,"cache_hit":0,"new_prefill":106394,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2028,"cache_hit":105984,"new_prefill":410,"score_linear":-105984.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2387,"cache_hit":0,"new_prefill":106394,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":3584,"new_prefill":102810,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":106394,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":3584,"new_prefill":102810,"score_linear":-3584.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":106394,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":106394,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-105984.0,"chosen_score_lmetric":0,"t_first_token":350000.483404217,"t_first_token_unix":1779780865.4973478,"t_done":350000.861170863,"t_done_unix":1779780865.8751144},{"request_id":"1399948:60:1732843:1065","session_id":"1399948","input_length":47074,"t_proxy_recv":350001.758511763,"t_decision_unix":1779780866.7724528,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":46592,"affinity_cache_ratio":0.9897608021413095,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":46592,"estimated_new_tokens":482,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1826,"cache_hit":0,"new_prefill":47074,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2028,"cache_hit":0,"new_prefill":47074,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2387,"cache_hit":46592,"new_prefill":482,"score_linear":-46592.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":0,"new_prefill":47074,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":47074,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":0,"new_prefill":47074,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":47074,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":6144,"new_prefill":40930,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-46592.0,"chosen_score_lmetric":0,"t_first_token":350001.87678558,"t_first_token_unix":1779780866.8907287,"t_done":350002.254648233,"t_done_unix":1779780867.268592},{"request_id":"1253804:46:1733935:1066","session_id":"1253804","input_length":106443,"t_proxy_recv":350005.259794849,"t_decision_unix":1779780870.2737362,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":105984,"affinity_cache_ratio":0.9956878329246639,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":105984,"estimated_new_tokens":459,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1826,"cache_hit":0,"new_prefill":106443,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2028,"cache_hit":105984,"new_prefill":459,"score_linear":-105984.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2388,"cache_hit":0,"new_prefill":106443,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":3584,"new_prefill":102859,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":106443,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":3584,"new_prefill":102859,"score_linear":-3584.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":106443,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":106443,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-105984.0,"chosen_score_lmetric":0,"t_first_token":350005.387530955,"t_first_token_unix":1779780870.4014742,"t_done":350006.108690466,"t_done_unix":1779780871.122634},{"request_id":"1399948:61:1735933:1067","session_id":"1399948","input_length":47205,"t_proxy_recv":350011.955469148,"t_decision_unix":1779780876.9694104,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":47104,"affinity_cache_ratio":0.9978603961444762,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":47104,"estimated_new_tokens":101,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1826,"cache_hit":0,"new_prefill":47205,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2029,"cache_hit":0,"new_prefill":47205,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2388,"cache_hit":47104,"new_prefill":101,"score_linear":-47104.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":0,"new_prefill":47205,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":47205,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":0,"new_prefill":47205,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":47205,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":6144,"new_prefill":41061,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-47104.0,"chosen_score_lmetric":0,"t_first_token":350012.055655283,"t_first_token_unix":1779780877.0695984,"t_done":350013.608767337,"t_done_unix":1779780878.6227105},{"request_id":"1253804:47:1736337:1068","session_id":"1253804","input_length":107137,"t_proxy_recv":350013.231201355,"t_decision_unix":1779780878.2451425,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":106496,"affinity_cache_ratio":0.994017006263009,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":106496,"estimated_new_tokens":641,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1826,"cache_hit":0,"new_prefill":107137,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2029,"cache_hit":106496,"new_prefill":641,"score_linear":-106496.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":47205,"ongoing_decode_tokens":47205,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2388,"cache_hit":0,"new_prefill":107137,"score_linear":47205.0,"score_lmetric":107137},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":3584,"new_prefill":103553,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":107137,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":3584,"new_prefill":103553,"score_linear":-3584.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":107137,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":107137,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-106496.0,"chosen_score_lmetric":0,"t_first_token":350013.990474963,"t_first_token_unix":1779780879.0044181,"t_done":350014.382775801,"t_done_unix":1779780879.3967195},{"request_id":"1253804:48:1737694:1069","session_id":"1253804","input_length":108145,"t_proxy_recv":350017.667038902,"t_decision_unix":1779780882.6809802,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":107008,"affinity_cache_ratio":0.98948633778723,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":107008,"estimated_new_tokens":1137,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1826,"cache_hit":0,"new_prefill":108145,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2030,"cache_hit":107008,"new_prefill":1137,"score_linear":-107008.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2388,"cache_hit":0,"new_prefill":108145,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":3584,"new_prefill":104561,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":108145,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":3584,"new_prefill":104561,"score_linear":-3584.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":108145,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":108145,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-107008.0,"chosen_score_lmetric":0,"t_first_token":350018.584887834,"t_first_token_unix":1779780883.5988317,"t_done":350019.011176938,"t_done_unix":1779780884.0251205},{"request_id":"1253804:49:1739030:1070","session_id":"1253804","input_length":109454,"t_proxy_recv":350022.053222035,"t_decision_unix":1779780887.0671632,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":108032,"affinity_cache_ratio":0.9870082409048551,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":108032,"estimated_new_tokens":1422,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1826,"cache_hit":0,"new_prefill":109454,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2032,"cache_hit":108032,"new_prefill":1422,"score_linear":-108032.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2388,"cache_hit":0,"new_prefill":109454,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":3584,"new_prefill":105870,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":109454,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":3584,"new_prefill":105870,"score_linear":-3584.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":109454,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":109454,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-108032.0,"chosen_score_lmetric":0,"t_first_token":350023.177719808,"t_first_token_unix":1779780888.191664,"t_done":350023.613263444,"t_done_unix":1779780888.6272073},{"request_id":"1253804:50:1741670:1071","session_id":"1253804","input_length":110854,"t_proxy_recv":350031.034442719,"t_decision_unix":1779780896.048384,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":109056,"affinity_cache_ratio":0.9837804680029588,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":109056,"estimated_new_tokens":1798,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1826,"cache_hit":0,"new_prefill":110854,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2034,"cache_hit":109056,"new_prefill":1798,"score_linear":-109056.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2388,"cache_hit":0,"new_prefill":110854,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":3584,"new_prefill":107270,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":110854,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":3584,"new_prefill":107270,"score_linear":-3584.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":110854,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":110854,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-109056.0,"chosen_score_lmetric":0,"t_first_token":350032.183156501,"t_first_token_unix":1779780897.1971,"t_done":350032.625439768,"t_done_unix":1779780897.639383},{"request_id":"1253804:51:1744514:1072","session_id":"1253804","input_length":111661,"t_proxy_recv":350040.461553314,"t_decision_unix":1779780905.4754946,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":110592,"affinity_cache_ratio":0.9904263798461415,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":110592,"estimated_new_tokens":1069,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1826,"cache_hit":0,"new_prefill":111661,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2037,"cache_hit":110592,"new_prefill":1069,"score_linear":-110592.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2388,"cache_hit":0,"new_prefill":111661,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":3584,"new_prefill":108077,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":111661,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":3584,"new_prefill":108077,"score_linear":-3584.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":111661,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":111661,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-110592.0,"chosen_score_lmetric":0,"t_first_token":350041.206321229,"t_first_token_unix":1779780906.2202652,"t_done":350041.645050143,"t_done_unix":1779780906.658994},{"request_id":"1340290:16:1745847:1073","session_id":"1340290","input_length":40200,"t_proxy_recv":350045.064232243,"t_decision_unix":1779780910.0781732,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":39936,"affinity_cache_ratio":0.9934328358208955,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":39936,"estimated_new_tokens":264,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1826,"cache_hit":0,"new_prefill":40200,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2039,"cache_hit":0,"new_prefill":40200,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2388,"cache_hit":39936,"new_prefill":264,"score_linear":-39936.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":0,"new_prefill":40200,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":40200,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":0,"new_prefill":40200,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":39176,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":40200,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-39936.0,"chosen_score_lmetric":0,"t_first_token":350045.139322711,"t_first_token_unix":1779780910.153266,"t_done":350045.871347671,"t_done_unix":1779780910.885291},{"request_id":"1253804:52:1746191:1074","session_id":"1253804","input_length":111724,"t_proxy_recv":350046.099737298,"t_decision_unix":1779780911.1136787,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":111616,"affinity_cache_ratio":0.9990333321399162,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":111616,"estimated_new_tokens":108,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1826,"cache_hit":0,"new_prefill":111724,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2039,"cache_hit":111616,"new_prefill":108,"score_linear":-111616.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2388,"cache_hit":0,"new_prefill":111724,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":3584,"new_prefill":108140,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":111724,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":3584,"new_prefill":108140,"score_linear":-3584.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":111724,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":111724,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-111616.0,"chosen_score_lmetric":0,"t_first_token":350046.249029652,"t_first_token_unix":1779780911.2629735,"t_done":350047.039105493,"t_done_unix":1779780912.053049},{"request_id":"1253804:53:1748377:1075","session_id":"1253804","input_length":111811,"t_proxy_recv":350053.436669537,"t_decision_unix":1779780918.4506106,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":111616,"affinity_cache_ratio":0.9982559855470392,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":111616,"estimated_new_tokens":195,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1826,"cache_hit":0,"new_prefill":111811,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2039,"cache_hit":111616,"new_prefill":195,"score_linear":-111616.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2388,"cache_hit":0,"new_prefill":111811,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":3584,"new_prefill":108227,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":111811,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":3584,"new_prefill":108227,"score_linear":-3584.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":111811,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":111811,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-111616.0,"chosen_score_lmetric":0,"t_first_token":350053.584864599,"t_first_token_unix":1779780918.5988078,"t_done":350054.082269477,"t_done_unix":1779780919.096213},{"request_id":"1399948:62:1749028:1076","session_id":"1399948","input_length":48212,"t_proxy_recv":350055.596104462,"t_decision_unix":1779780920.6100457,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":47104,"affinity_cache_ratio":0.9770181697502697,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":47104,"estimated_new_tokens":1108,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1826,"cache_hit":0,"new_prefill":48212,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2039,"cache_hit":0,"new_prefill":48212,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2388,"cache_hit":47104,"new_prefill":1108,"score_linear":-47104.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":0,"new_prefill":48212,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":48212,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":0,"new_prefill":48212,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":48212,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":6144,"new_prefill":42068,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-47104.0,"chosen_score_lmetric":0,"t_first_token":350055.956831925,"t_first_token_unix":1779780920.9707754,"t_done":350056.517694495,"t_done_unix":1779780921.5316374},{"request_id":"1253804:54:1750056:1077","session_id":"1253804","input_length":112074,"t_proxy_recv":350059.028597975,"t_decision_unix":1779780924.0425396,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":111616,"affinity_cache_ratio":0.9959134143512323,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":111616,"estimated_new_tokens":458,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1826,"cache_hit":0,"new_prefill":112074,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2039,"cache_hit":111616,"new_prefill":458,"score_linear":-111616.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2390,"cache_hit":0,"new_prefill":112074,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":3584,"new_prefill":108490,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":112074,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":3584,"new_prefill":108490,"score_linear":-3584.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":112074,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":112074,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-111616.0,"chosen_score_lmetric":0,"t_first_token":350059.362874766,"t_first_token_unix":1779780924.376818,"t_done":350065.320779522,"t_done_unix":1779780930.3347228},{"request_id":"1399948:63:1753316:1079","session_id":"1399948","input_length":48636,"t_proxy_recv":350069.894467184,"t_decision_unix":1779780934.9084086,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":48128,"affinity_cache_ratio":0.9895550620939222,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":48128,"estimated_new_tokens":508,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":133552,"ongoing_decode_tokens":0,"pending_prefill_tokens":432,"num_requests":1,"active_p_offloads":0,"cached_blocks":1826,"cache_hit":0,"new_prefill":48636,"score_linear":133552.0,"score_lmetric":49068},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2040,"cache_hit":0,"new_prefill":48636,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2390,"cache_hit":48128,"new_prefill":508,"score_linear":-48128.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":0,"new_prefill":48636,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":48636,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":0,"new_prefill":48636,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":48636,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":6144,"new_prefill":42492,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-48128.0,"chosen_score_lmetric":0,"t_first_token":350070.153773427,"t_first_token_unix":1779780935.1677165,"t_done":350070.265790466,"t_done_unix":1779780935.2797337},{"request_id":"1399948:64:1754453:1080","session_id":"1399948","input_length":48693,"t_proxy_recv":350073.807799218,"t_decision_unix":1779780938.8217404,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":48640,"affinity_cache_ratio":0.9989115478610888,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":48640,"estimated_new_tokens":53,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":133552,"ongoing_decode_tokens":133552,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1826,"cache_hit":0,"new_prefill":48693,"score_linear":133552.0,"score_lmetric":48693},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2040,"cache_hit":0,"new_prefill":48693,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2391,"cache_hit":48640,"new_prefill":53,"score_linear":-48640.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":0,"new_prefill":48693,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":48693,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":0,"new_prefill":48693,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":48693,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":6144,"new_prefill":42549,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-48640.0,"chosen_score_lmetric":0,"t_first_token":350073.917447886,"t_first_token_unix":1779780938.931391,"t_done":350075.21270882,"t_done_unix":1779780940.226652},{"request_id":"1356885:11:1753167:1078","session_id":"1356885","input_length":133552,"t_proxy_recv":350069.467718122,"t_decision_unix":1779780934.4816594,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":133120,"affinity_cache_ratio":0.9967653048999641,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":133120,"estimated_new_tokens":432,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1826,"cache_hit":133120,"new_prefill":432,"score_linear":-133120.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2040,"cache_hit":1024,"new_prefill":132528,"score_linear":-1024.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2390,"cache_hit":0,"new_prefill":133552,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":0,"new_prefill":133552,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":6656,"new_prefill":126896,"score_linear":-6656.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":0,"new_prefill":133552,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":133552,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":133552,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-133120.0,"chosen_score_lmetric":0,"t_first_token":350070.108036394,"t_first_token_unix":1779780935.1219802,"t_done":350087.44328248,"t_done_unix":1779780952.457226},{"request_id":"1399948:65:1759200:1081","session_id":"1399948","input_length":49220,"t_proxy_recv":350090.302079935,"t_decision_unix":1779780955.316021,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":48640,"affinity_cache_ratio":0.9882161722876879,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":48640,"estimated_new_tokens":580,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1827,"cache_hit":0,"new_prefill":49220,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2040,"cache_hit":0,"new_prefill":49220,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2391,"cache_hit":48640,"new_prefill":580,"score_linear":-48640.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":0,"new_prefill":49220,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":49220,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":0,"new_prefill":49220,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":49220,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":6144,"new_prefill":43076,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-48640.0,"chosen_score_lmetric":0,"t_first_token":350090.567238583,"t_first_token_unix":1779780955.581182,"t_done":350096.183059052,"t_done_unix":1779780961.1970022},{"request_id":"1277428:14:1767578:1082","session_id":"1277428","input_length":49716,"t_proxy_recv":350119.163735805,"t_decision_unix":1779780984.1776772,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":49152,"affinity_cache_ratio":0.9886555636012552,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":49152,"estimated_new_tokens":564,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1827,"cache_hit":0,"new_prefill":49716,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2040,"cache_hit":4608,"new_prefill":45108,"score_linear":-4608.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2393,"cache_hit":0,"new_prefill":49716,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":17408,"new_prefill":32308,"score_linear":-17408.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":49716,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1762,"cache_hit":49152,"new_prefill":564,"score_linear":-49152.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":49716,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":49716,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-49152.0,"chosen_score_lmetric":0,"t_first_token":350119.340969088,"t_first_token_unix":1779780984.3549123,"t_done":350120.389666582,"t_done_unix":1779780985.4036098},{"request_id":"1277428:14:1770237:1083","session_id":"1277428","input_length":49925,"t_proxy_recv":350128.510473807,"t_decision_unix":1779780993.5244145,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":49664,"affinity_cache_ratio":0.994772158237356,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":49664,"estimated_new_tokens":261,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1827,"cache_hit":0,"new_prefill":49925,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2040,"cache_hit":4608,"new_prefill":45317,"score_linear":-4608.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2393,"cache_hit":0,"new_prefill":49925,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":17408,"new_prefill":32517,"score_linear":-17408.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":49925,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1763,"cache_hit":49664,"new_prefill":261,"score_linear":-49664.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":49925,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":49925,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-49664.0,"chosen_score_lmetric":0,"t_first_token":350128.618642041,"t_first_token_unix":1779780993.6325853,"t_done":350128.935522079,"t_done_unix":1779780993.9494653},{"request_id":"1277428:15:1771467:1084","session_id":"1277428","input_length":51398,"t_proxy_recv":350132.580411047,"t_decision_unix":1779780997.5943527,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":49664,"affinity_cache_ratio":0.9662632787267987,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":49664,"estimated_new_tokens":1734,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1827,"cache_hit":0,"new_prefill":51398,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2040,"cache_hit":4608,"new_prefill":46790,"score_linear":-4608.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2393,"cache_hit":0,"new_prefill":51398,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":17408,"new_prefill":33990,"score_linear":-17408.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":51398,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1763,"cache_hit":49664,"new_prefill":1734,"score_linear":-49664.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":51398,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":51398,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-49664.0,"chosen_score_lmetric":0,"t_first_token":350133.172269638,"t_first_token_unix":1779780998.1862135,"t_done":350136.673569056,"t_done_unix":1779781001.6875126},{"request_id":"1399948:66:1772336:1085","session_id":"1399948","input_length":51012,"t_proxy_recv":350135.528757712,"t_decision_unix":1779781000.542699,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":49664,"affinity_cache_ratio":0.9735748451344781,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":49664,"estimated_new_tokens":1348,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1827,"cache_hit":0,"new_prefill":51012,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2040,"cache_hit":0,"new_prefill":51012,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2393,"cache_hit":49664,"new_prefill":1348,"score_linear":-49664.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":0,"new_prefill":51012,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":51012,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":51398,"ongoing_decode_tokens":51398,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1763,"cache_hit":0,"new_prefill":51012,"score_linear":51398.0,"score_lmetric":51012},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":51012,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":6144,"new_prefill":44868,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-49664.0,"chosen_score_lmetric":0,"t_first_token":350136.004965163,"t_first_token_unix":1779781001.0189092,"t_done":350138.993819182,"t_done_unix":1779781004.0077627},{"request_id":"1399948:67:1777074:1086","session_id":"1399948","input_length":52331,"t_proxy_recv":350151.597856074,"t_decision_unix":1779781016.6117976,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":51200,"affinity_cache_ratio":0.9783875714203818,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":51200,"estimated_new_tokens":1131,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1827,"cache_hit":0,"new_prefill":52331,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2040,"cache_hit":0,"new_prefill":52331,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2396,"cache_hit":51200,"new_prefill":1131,"score_linear":-51200.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":0,"new_prefill":52331,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":52331,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1767,"cache_hit":0,"new_prefill":52331,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":52331,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":6144,"new_prefill":46187,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-51200.0,"chosen_score_lmetric":0,"t_first_token":350152.075298637,"t_first_token_unix":1779781017.0892422,"t_done":350156.433904288,"t_done_unix":1779781021.4478476},{"request_id":"1399948:68:1784799:1087","session_id":"1399948","input_length":53289,"t_proxy_recv":350178.086887202,"t_decision_unix":1779781043.1008284,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":52736,"affinity_cache_ratio":0.9896226238060387,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":52736,"estimated_new_tokens":553,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1827,"cache_hit":0,"new_prefill":53289,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2040,"cache_hit":0,"new_prefill":53289,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2399,"cache_hit":52736,"new_prefill":553,"score_linear":-52736.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":0,"new_prefill":53289,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":53289,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1767,"cache_hit":0,"new_prefill":53289,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":53289,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":6144,"new_prefill":47145,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-52736.0,"chosen_score_lmetric":0,"t_first_token":350178.37075088,"t_first_token_unix":1779781043.384694,"t_done":350182.716352415,"t_done_unix":1779781047.7302961},{"request_id":"1277428:16:1787662:1088","session_id":"1277428","input_length":52754,"t_proxy_recv":350188.53396327,"t_decision_unix":1779781053.547904,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":51712,"affinity_cache_ratio":0.9802479432839216,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":51712,"estimated_new_tokens":1042,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1827,"cache_hit":0,"new_prefill":52754,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2040,"cache_hit":4608,"new_prefill":48146,"score_linear":-4608.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2401,"cache_hit":0,"new_prefill":52754,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":17408,"new_prefill":35346,"score_linear":-17408.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":52754,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1767,"cache_hit":51712,"new_prefill":1042,"score_linear":-51712.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":52754,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":52754,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-51712.0,"chosen_score_lmetric":0,"t_first_token":350188.940730754,"t_first_token_unix":1779781053.9546745,"t_done":350189.53143809,"t_done_unix":1779781054.5453813},{"request_id":"1305906:5:1788459:1089","session_id":"1305906","input_length":52461,"t_proxy_recv":350191.068950377,"t_decision_unix":1779781056.0828915,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":49664,"affinity_cache_ratio":0.946684203503555,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":49664,"estimated_new_tokens":2797,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1827,"cache_hit":0,"new_prefill":52461,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2040,"cache_hit":0,"new_prefill":52461,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2401,"cache_hit":49664,"new_prefill":2797,"score_linear":-49664.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":0,"new_prefill":52461,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":52461,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1769,"cache_hit":0,"new_prefill":52461,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":52461,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":52461,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-49664.0,"chosen_score_lmetric":0,"t_first_token":350192.054742111,"t_first_token_unix":1779781057.0686855,"t_done":350192.570112671,"t_done_unix":1779781057.5840561},{"request_id":"1399948:69:1790968:1090","session_id":"1399948","input_length":54141,"t_proxy_recv":350199.673542605,"t_decision_unix":1779781064.6874838,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":53760,"affinity_cache_ratio":0.9929628193051476,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":53760,"estimated_new_tokens":381,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1827,"cache_hit":0,"new_prefill":54141,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2040,"cache_hit":0,"new_prefill":54141,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":51255,"ongoing_decode_tokens":51255,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2406,"cache_hit":53760,"new_prefill":381,"score_linear":-2505.0,"score_lmetric":381},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":0,"new_prefill":54141,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":54141,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1769,"cache_hit":0,"new_prefill":54141,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":54141,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":6144,"new_prefill":47997,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-2505.0,"chosen_score_lmetric":381,"t_first_token":350199.953422225,"t_first_token_unix":1779781064.9673662,"t_done":350205.765676327,"t_done_unix":1779781070.7796197},{"request_id":"1305906:6:1529600:945","session_id":"1305906","input_length":51255,"t_proxy_recv":350192.580254204,"t_decision_unix":1779781057.594196,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":51200,"affinity_cache_ratio":0.9989269339576626,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":51200,"estimated_new_tokens":55,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1827,"cache_hit":0,"new_prefill":51255,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2040,"cache_hit":0,"new_prefill":51255,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2406,"cache_hit":51200,"new_prefill":55,"score_linear":-51200.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":0,"new_prefill":51255,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":51255,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1769,"cache_hit":0,"new_prefill":51255,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":51255,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":51255,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-51200.0,"chosen_score_lmetric":0,"t_first_token":350192.653793864,"t_first_token_unix":1779781057.667737,"t_done":350209.906840258,"t_done_unix":1779781074.9207838},{"request_id":"1277428:17:1793546:1091","session_id":"1277428","input_length":53574,"t_proxy_recv":350208.664915773,"t_decision_unix":1779781073.6788568,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":52736,"affinity_cache_ratio":0.9843580841452944,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":52736,"estimated_new_tokens":838,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1827,"cache_hit":0,"new_prefill":53574,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2040,"cache_hit":4608,"new_prefill":48966,"score_linear":-4608.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":51255,"ongoing_decode_tokens":51255,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2407,"cache_hit":0,"new_prefill":53574,"score_linear":51255.0,"score_lmetric":53574},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":17408,"new_prefill":36166,"score_linear":-17408.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":53574,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1769,"cache_hit":52736,"new_prefill":838,"score_linear":-52736.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":53574,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":53574,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-52736.0,"chosen_score_lmetric":0,"t_first_token":350209.061762005,"t_first_token_unix":1779781074.0757053,"t_done":350210.386432611,"t_done_unix":1779781075.4003758},{"request_id":"1399948:70:1795589:1092","session_id":"1399948","input_length":54654,"t_proxy_recv":350215.991378386,"t_decision_unix":1779781081.0053196,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":54272,"affinity_cache_ratio":0.9930105756211806,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":54272,"estimated_new_tokens":382,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1827,"cache_hit":0,"new_prefill":54654,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2040,"cache_hit":0,"new_prefill":54654,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2409,"cache_hit":54272,"new_prefill":382,"score_linear":-54272.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":0,"new_prefill":54654,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":54654,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1770,"cache_hit":0,"new_prefill":54654,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":54654,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":6144,"new_prefill":48510,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-54272.0,"chosen_score_lmetric":0,"t_first_token":350216.084046648,"t_first_token_unix":1779781081.0979905,"t_done":350217.026392553,"t_done_unix":1779781082.0403361},{"request_id":"1277428:18:1796824:1093","session_id":"1277428","input_length":53805,"t_proxy_recv":350220.589630781,"t_decision_unix":1779781085.603572,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":53248,"affinity_cache_ratio":0.9896478022488616,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":53248,"estimated_new_tokens":557,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1827,"cache_hit":0,"new_prefill":53805,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2040,"cache_hit":4608,"new_prefill":49197,"score_linear":-4608.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2409,"cache_hit":0,"new_prefill":53805,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":17408,"new_prefill":36397,"score_linear":-17408.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":53805,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1770,"cache_hit":53248,"new_prefill":557,"score_linear":-53248.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":53805,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":53805,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-53248.0,"chosen_score_lmetric":0,"t_first_token":350220.716822817,"t_first_token_unix":1779781085.730766,"t_done":350221.094891266,"t_done_unix":1779781086.1088347},{"request_id":"1399948:71:1800797:1094","session_id":"1399948","input_length":54920,"t_proxy_recv":350234.172524312,"t_decision_unix":1779781099.1864657,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":54272,"affinity_cache_ratio":0.9882010196649672,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":54272,"estimated_new_tokens":648,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1827,"cache_hit":0,"new_prefill":54920,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2040,"cache_hit":0,"new_prefill":54920,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2409,"cache_hit":54272,"new_prefill":648,"score_linear":-54272.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":0,"new_prefill":54920,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":54920,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1771,"cache_hit":0,"new_prefill":54920,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":54920,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":6144,"new_prefill":48776,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-54272.0,"chosen_score_lmetric":0,"t_first_token":350234.355387703,"t_first_token_unix":1779781099.369331,"t_done":350236.448671998,"t_done_unix":1779781101.4626153},{"request_id":"1399948:72:1804699:1097","session_id":"1399948","input_length":55246,"t_proxy_recv":350247.623829684,"t_decision_unix":1779781112.637771,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":54784,"affinity_cache_ratio":0.9916374036129313,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":54784,"estimated_new_tokens":462,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1827,"cache_hit":0,"new_prefill":55246,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112634,"ongoing_decode_tokens":112634,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2040,"cache_hit":0,"new_prefill":55246,"score_linear":112634.0,"score_lmetric":55246},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2410,"cache_hit":54784,"new_prefill":462,"score_linear":-54784.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":111736,"ongoing_decode_tokens":111736,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":0,"new_prefill":55246,"score_linear":111736.0,"score_lmetric":55246},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":55246,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1771,"cache_hit":0,"new_prefill":55246,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":55246,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":6144,"new_prefill":49102,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-54784.0,"chosen_score_lmetric":0,"t_first_token":350247.74131986,"t_first_token_unix":1779781112.7552633,"t_done":350248.13906601,"t_done_unix":1779781113.1530097},{"request_id":"1399948:73:1805981:1098","session_id":"1399948","input_length":55559,"t_proxy_recv":350251.805550858,"t_decision_unix":1779781116.819492,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":55296,"affinity_cache_ratio":0.9952662934898037,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":55296,"estimated_new_tokens":263,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1827,"cache_hit":0,"new_prefill":55559,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112634,"ongoing_decode_tokens":112634,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2040,"cache_hit":0,"new_prefill":55559,"score_linear":112634.0,"score_lmetric":55559},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2411,"cache_hit":55296,"new_prefill":263,"score_linear":-55296.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":111736,"ongoing_decode_tokens":111736,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":0,"new_prefill":55559,"score_linear":111736.0,"score_lmetric":55559},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":55559,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1771,"cache_hit":0,"new_prefill":55559,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":55559,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":6144,"new_prefill":49415,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-55296.0,"chosen_score_lmetric":0,"t_first_token":350251.997953132,"t_first_token_unix":1779781117.0119004,"t_done":350253.297022642,"t_done_unix":1779781118.3109663},{"request_id":"1305906:6:1806419:1099","session_id":"1305906","input_length":52932,"t_proxy_recv":350253.315457534,"t_decision_unix":1779781118.3293989,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":52736,"affinity_cache_ratio":0.9962971359480087,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":52736,"estimated_new_tokens":196,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1827,"cache_hit":0,"new_prefill":52932,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112634,"ongoing_decode_tokens":112634,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2040,"cache_hit":0,"new_prefill":52932,"score_linear":112634.0,"score_lmetric":52932},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2411,"cache_hit":52736,"new_prefill":196,"score_linear":-52736.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":111736,"ongoing_decode_tokens":111736,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":0,"new_prefill":52932,"score_linear":111736.0,"score_lmetric":52932},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":52932,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1771,"cache_hit":0,"new_prefill":52932,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":52932,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":52932,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-52736.0,"chosen_score_lmetric":0,"t_first_token":350253.384842867,"t_first_token_unix":1779781118.3987863,"t_done":350254.104591445,"t_done_unix":1779781119.1185343},{"request_id":"1305906:7:1572780:964","session_id":"1305906","input_length":54789,"t_proxy_recv":350254.115346662,"t_decision_unix":1779781119.1292884,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":52736,"affinity_cache_ratio":0.9625289747942105,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":52736,"estimated_new_tokens":2053,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1827,"cache_hit":0,"new_prefill":54789,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112634,"ongoing_decode_tokens":112634,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2040,"cache_hit":0,"new_prefill":54789,"score_linear":112634.0,"score_lmetric":54789},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2411,"cache_hit":52736,"new_prefill":2053,"score_linear":-52736.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":111736,"ongoing_decode_tokens":111736,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":0,"new_prefill":54789,"score_linear":111736.0,"score_lmetric":54789},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":54789,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1771,"cache_hit":0,"new_prefill":54789,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":54789,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":54789,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-52736.0,"chosen_score_lmetric":0,"t_first_token":350254.832392296,"t_first_token_unix":1779781119.846336,"t_done":350255.139974368,"t_done_unix":1779781120.1539185},{"request_id":"1399948:74:1808473:1100","session_id":"1399948","input_length":55791,"t_proxy_recv":350260.506698216,"t_decision_unix":1779781125.5206394,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":55296,"affinity_cache_ratio":0.9911276012260042,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":55296,"estimated_new_tokens":495,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1827,"cache_hit":0,"new_prefill":55791,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112634,"ongoing_decode_tokens":112634,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2040,"cache_hit":0,"new_prefill":55791,"score_linear":112634.0,"score_lmetric":55791},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2415,"cache_hit":55296,"new_prefill":495,"score_linear":-55296.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":111736,"ongoing_decode_tokens":111736,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":0,"new_prefill":55791,"score_linear":111736.0,"score_lmetric":55791},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":55791,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1771,"cache_hit":0,"new_prefill":55791,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":55791,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":6144,"new_prefill":49647,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-55296.0,"chosen_score_lmetric":0,"t_first_token":350260.638611399,"t_first_token_unix":1779781125.652555,"t_done":350262.527207562,"t_done_unix":1779781127.541151},{"request_id":"1270606:6:1804345:1096","session_id":"1270606","input_length":111736,"t_proxy_recv":350246.491666525,"t_decision_unix":1779781111.5056078,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":111616,"affinity_cache_ratio":0.9989260399513138,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":111616,"estimated_new_tokens":120,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1827,"cache_hit":0,"new_prefill":111736,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112634,"ongoing_decode_tokens":112634,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2040,"cache_hit":0,"new_prefill":111736,"score_linear":112634.0,"score_lmetric":111736},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2410,"cache_hit":0,"new_prefill":111736,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":111616,"new_prefill":120,"score_linear":-111616.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":1536,"new_prefill":110200,"score_linear":-1536.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1771,"cache_hit":0,"new_prefill":111736,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":111736,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":111736,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-111616.0,"chosen_score_lmetric":0,"t_first_token":350246.775907002,"t_first_token_unix":1779781111.7898512,"t_done":350289.315690738,"t_done_unix":1779781154.3296347},{"request_id":"1399948:75:1817312:1101","session_id":"1399948","input_length":57308,"t_proxy_recv":350289.048119311,"t_decision_unix":1779781154.0620604,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":55808,"affinity_cache_ratio":0.9738256438891604,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":55808,"estimated_new_tokens":1500,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1827,"cache_hit":0,"new_prefill":57308,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112634,"ongoing_decode_tokens":112634,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2040,"cache_hit":0,"new_prefill":57308,"score_linear":112634.0,"score_lmetric":57308},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2416,"cache_hit":55808,"new_prefill":1500,"score_linear":-55808.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":111736,"ongoing_decode_tokens":111736,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":0,"new_prefill":57308,"score_linear":111736.0,"score_lmetric":57308},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":57308,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1771,"cache_hit":0,"new_prefill":57308,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":57308,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":6144,"new_prefill":51164,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-55808.0,"chosen_score_lmetric":0,"t_first_token":350289.680174747,"t_first_token_unix":1779781154.6941185,"t_done":350290.522926003,"t_done_unix":1779781155.536869},{"request_id":"1399948:76:1821495:1102","session_id":"1399948","input_length":57659,"t_proxy_recv":350302.701180417,"t_decision_unix":1779781167.7151215,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":57344,"affinity_cache_ratio":0.9945368459390554,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":57344,"estimated_new_tokens":315,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1827,"cache_hit":0,"new_prefill":57659,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112634,"ongoing_decode_tokens":112634,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2040,"cache_hit":0,"new_prefill":57659,"score_linear":112634.0,"score_lmetric":57659},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2419,"cache_hit":57344,"new_prefill":315,"score_linear":-57344.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2223,"cache_hit":0,"new_prefill":57659,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":57659,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1771,"cache_hit":0,"new_prefill":57659,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":57659,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":6144,"new_prefill":51515,"score_linear":-6144.0,"score_lmetric":0}],"chosen_score_linear":-57344.0,"chosen_score_lmetric":0,"t_first_token":350302.894226709,"t_first_token_unix":1779781167.9081702,"t_done":350303.559867932,"t_done_unix":1779781168.5738113},{"request_id":"1270606:6:1822681:1103","session_id":"1270606","input_length":112180,"t_proxy_recv":350306.542647085,"t_decision_unix":1779781171.556588,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":112128,"affinity_cache_ratio":0.9995364592619005,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":112128,"estimated_new_tokens":52,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1827,"cache_hit":0,"new_prefill":112180,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112634,"ongoing_decode_tokens":112634,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2040,"cache_hit":0,"new_prefill":112180,"score_linear":112634.0,"score_lmetric":112180},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2419,"cache_hit":0,"new_prefill":112180,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2223,"cache_hit":112128,"new_prefill":52,"score_linear":-112128.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":1536,"new_prefill":110644,"score_linear":-1536.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1771,"cache_hit":0,"new_prefill":112180,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":112180,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":112180,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-112128.0,"chosen_score_lmetric":0,"t_first_token":350306.682014508,"t_first_token_unix":1779781171.695958,"t_done":350307.574405165,"t_done_unix":1779781172.5883493},{"request_id":"1270606:7:1825806:1104","session_id":"1270606","input_length":112645,"t_proxy_recv":350317.161322781,"t_decision_unix":1779781182.175264,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":112128,"affinity_cache_ratio":0.9954103599804697,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":112128,"estimated_new_tokens":517,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1827,"cache_hit":0,"new_prefill":112645,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112634,"ongoing_decode_tokens":112634,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2040,"cache_hit":0,"new_prefill":112645,"score_linear":112634.0,"score_lmetric":112645},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2419,"cache_hit":0,"new_prefill":112645,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2223,"cache_hit":112128,"new_prefill":517,"score_linear":-112128.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":1536,"new_prefill":111109,"score_linear":-1536.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1771,"cache_hit":0,"new_prefill":112645,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":112645,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":112645,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-112128.0,"chosen_score_lmetric":0,"t_first_token":350317.738436581,"t_first_token_unix":1779781182.752381,"t_done":350321.768240846,"t_done_unix":1779781186.782185},{"request_id":"1277428:19:1834491:1105","session_id":"1277428","input_length":58474,"t_proxy_recv":350346.428191398,"t_decision_unix":1779781211.442131,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":53760,"affinity_cache_ratio":0.9193829736293053,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":53760,"estimated_new_tokens":4714,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1827,"cache_hit":0,"new_prefill":58474,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112634,"ongoing_decode_tokens":112634,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2040,"cache_hit":4608,"new_prefill":53866,"score_linear":108026.0,"score_lmetric":53866},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2419,"cache_hit":0,"new_prefill":58474,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2224,"cache_hit":17408,"new_prefill":41066,"score_linear":-17408.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":58474,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1771,"cache_hit":53760,"new_prefill":4714,"score_linear":-53760.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":58474,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":58474,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-53760.0,"chosen_score_lmetric":0,"t_first_token":350348.164266275,"t_first_token_unix":1779781213.17821,"t_done":350348.50253058,"t_done_unix":1779781213.516474},{"request_id":"1270606:8:1837578:1106","session_id":"1270606","input_length":113482,"t_proxy_recv":350357.259721828,"t_decision_unix":1779781222.273663,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":112640,"affinity_cache_ratio":0.9925803211081934,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":112640,"estimated_new_tokens":842,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1827,"cache_hit":0,"new_prefill":113482,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112634,"ongoing_decode_tokens":112634,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2040,"cache_hit":0,"new_prefill":113482,"score_linear":112634.0,"score_lmetric":113482},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2419,"cache_hit":0,"new_prefill":113482,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2224,"cache_hit":112640,"new_prefill":842,"score_linear":-112640.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":1536,"new_prefill":111946,"score_linear":-1536.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1780,"cache_hit":0,"new_prefill":113482,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":113482,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":113482,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-112640.0,"chosen_score_lmetric":0,"t_first_token":350357.821723502,"t_first_token_unix":1779781222.8356674,"t_done":350358.250376026,"t_done_unix":1779781223.2643242},{"request_id":"1253804:55:1803044:1095","session_id":"1253804","input_length":112634,"t_proxy_recv":350242.049776928,"t_decision_unix":1779781107.063718,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":112128,"affinity_cache_ratio":0.9955075732016976,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":112128,"estimated_new_tokens":506,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1827,"cache_hit":0,"new_prefill":112634,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2040,"cache_hit":112128,"new_prefill":506,"score_linear":-112128.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2410,"cache_hit":0,"new_prefill":112634,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2221,"cache_hit":3584,"new_prefill":109050,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":112634,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1771,"cache_hit":3584,"new_prefill":109050,"score_linear":-3584.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":112634,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":112634,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-112128.0,"chosen_score_lmetric":0,"t_first_token":350242.215928737,"t_first_token_unix":1779781107.229872,"t_done":350367.160874663,"t_done_unix":1779781232.1748185},{"request_id":"1270606:9:1847569:1107","session_id":"1270606","input_length":116467,"t_proxy_recv":350391.163036484,"t_decision_unix":1779781256.1769779,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":113152,"affinity_cache_ratio":0.9715370018975332,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":113152,"estimated_new_tokens":3315,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1827,"cache_hit":0,"new_prefill":116467,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":116467,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2419,"cache_hit":0,"new_prefill":116467,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2225,"cache_hit":113152,"new_prefill":3315,"score_linear":-113152.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":1536,"new_prefill":114931,"score_linear":-1536.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1780,"cache_hit":0,"new_prefill":116467,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":116467,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":116467,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-113152.0,"chosen_score_lmetric":0,"t_first_token":350393.407803204,"t_first_token_unix":1779781258.4217484,"t_done":350394.947527731,"t_done_unix":1779781259.961471},{"request_id":"1270606:10:1851128:1108","session_id":"1270606","input_length":116659,"t_proxy_recv":350403.246716278,"t_decision_unix":1779781268.2606573,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":116224,"affinity_cache_ratio":0.9962711835349181,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":116224,"estimated_new_tokens":435,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1827,"cache_hit":0,"new_prefill":116659,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":116659,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2419,"cache_hit":0,"new_prefill":116659,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2231,"cache_hit":116224,"new_prefill":435,"score_linear":-116224.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":1536,"new_prefill":115123,"score_linear":-1536.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1780,"cache_hit":0,"new_prefill":116659,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":116659,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":116659,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-116224.0,"chosen_score_lmetric":0,"t_first_token":350403.499425688,"t_first_token_unix":1779781268.513369,"t_done":350404.251801385,"t_done_unix":1779781269.2657452},{"request_id":"1305906:7:1851943:1109","session_id":"1305906","input_length":58042,"t_proxy_recv":350405.791360661,"t_decision_unix":1779781270.805302,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":54784,"affinity_cache_ratio":0.9438682333482651,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":54784,"estimated_new_tokens":3258,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1827,"cache_hit":0,"new_prefill":58042,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":58042,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2419,"cache_hit":54784,"new_prefill":3258,"score_linear":-54784.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2231,"cache_hit":0,"new_prefill":58042,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":58042,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1780,"cache_hit":0,"new_prefill":58042,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":58042,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":58042,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-54784.0,"chosen_score_lmetric":0,"t_first_token":350407.086499546,"t_first_token_unix":1779781272.1004431,"t_done":350409.362888506,"t_done_unix":1779781274.3768318},{"request_id":"1305906:8:1866131:1110","session_id":"1305906","input_length":58687,"t_proxy_recv":350454.297097758,"t_decision_unix":1779781319.3110387,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":57856,"affinity_cache_ratio":0.9858401349532264,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":57856,"estimated_new_tokens":831,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1827,"cache_hit":0,"new_prefill":58687,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":58687,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2425,"cache_hit":57856,"new_prefill":831,"score_linear":-57856.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2231,"cache_hit":0,"new_prefill":58687,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":58687,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1780,"cache_hit":0,"new_prefill":58687,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":58687,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":58687,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-57856.0,"chosen_score_lmetric":0,"t_first_token":350454.607381439,"t_first_token_unix":1779781319.6213248,"t_done":350456.507363761,"t_done_unix":1779781321.5213077},{"request_id":"1270606:11:1868274:1111","session_id":"1270606","input_length":117828,"t_proxy_recv":350461.527628331,"t_decision_unix":1779781326.541568,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":116224,"affinity_cache_ratio":0.9863869368910615,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":116224,"estimated_new_tokens":1604,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1827,"cache_hit":0,"new_prefill":117828,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":117828,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2427,"cache_hit":0,"new_prefill":117828,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2231,"cache_hit":116224,"new_prefill":1604,"score_linear":-116224.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":1536,"new_prefill":116292,"score_linear":-1536.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1780,"cache_hit":0,"new_prefill":117828,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":117828,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":117828,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-116224.0,"chosen_score_lmetric":0,"t_first_token":350462.589785885,"t_first_token_unix":1779781327.6037295,"t_done":350464.276272365,"t_done_unix":1779781329.290216},{"request_id":"1270606:11:1870810:1113","session_id":"1270606","input_length":118012,"t_proxy_recv":350470.026252455,"t_decision_unix":1779781335.0401938,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":117760,"affinity_cache_ratio":0.9978646239365488,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":117760,"estimated_new_tokens":252,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1827,"cache_hit":0,"new_prefill":118012,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112926,"ongoing_decode_tokens":112926,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":118012,"score_linear":112926.0,"score_lmetric":118012},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2427,"cache_hit":0,"new_prefill":118012,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2234,"cache_hit":117760,"new_prefill":252,"score_linear":-117760.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":1536,"new_prefill":116476,"score_linear":-1536.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1780,"cache_hit":0,"new_prefill":118012,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":118012,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":118012,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-117760.0,"chosen_score_lmetric":0,"t_first_token":350470.214912477,"t_first_token_unix":1779781335.2288566,"t_done":350470.711867987,"t_done_unix":1779781335.7258117},{"request_id":"1356885:12:1874205:1114","session_id":"1356885","input_length":135435,"t_proxy_recv":350481.499382041,"t_decision_unix":1779781346.5133233,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":134656,"affinity_cache_ratio":0.9942481633255805,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":134656,"estimated_new_tokens":779,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1827,"cache_hit":134656,"new_prefill":779,"score_linear":-134656.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112926,"ongoing_decode_tokens":112926,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":1024,"new_prefill":134411,"score_linear":111902.0,"score_lmetric":134411},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2427,"cache_hit":0,"new_prefill":135435,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2234,"cache_hit":0,"new_prefill":135435,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":6656,"new_prefill":128779,"score_linear":-6656.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1780,"cache_hit":0,"new_prefill":135435,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":135435,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":135435,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-134656.0,"chosen_score_lmetric":0,"t_first_token":350482.155635435,"t_first_token_unix":1779781347.1695793,"t_done":350484.71270422,"t_done_unix":1779781349.7266479},{"request_id":"1340290:17:1875101:1115","session_id":"1340290","input_length":40328,"t_proxy_recv":350484.422109895,"t_decision_unix":1779781349.4360511,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":39936,"affinity_cache_ratio":0.9902797064074589,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":39936,"estimated_new_tokens":392,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135435,"ongoing_decode_tokens":135435,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1827,"cache_hit":0,"new_prefill":40328,"score_linear":135435.0,"score_lmetric":40328},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112926,"ongoing_decode_tokens":112926,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":40328,"score_linear":112926.0,"score_lmetric":40328},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2427,"cache_hit":39936,"new_prefill":392,"score_linear":-39936.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2234,"cache_hit":0,"new_prefill":40328,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":40328,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1780,"cache_hit":0,"new_prefill":40328,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":39304,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":40328,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-39936.0,"chosen_score_lmetric":0,"t_first_token":350484.49356925,"t_first_token_unix":1779781349.5075226,"t_done":350487.631724709,"t_done_unix":1779781352.645668},{"request_id":"1270606:12:1876851:1116","session_id":"1270606","input_length":118600,"t_proxy_recv":350490.388614283,"t_decision_unix":1779781355.4025555,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":117760,"affinity_cache_ratio":0.9929173693086003,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":117760,"estimated_new_tokens":840,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1828,"cache_hit":0,"new_prefill":118600,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112926,"ongoing_decode_tokens":112926,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":118600,"score_linear":112926.0,"score_lmetric":118600},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2428,"cache_hit":0,"new_prefill":118600,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2234,"cache_hit":117760,"new_prefill":840,"score_linear":-117760.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":1536,"new_prefill":117064,"score_linear":-1536.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1780,"cache_hit":0,"new_prefill":118600,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":118600,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":118600,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-117760.0,"chosen_score_lmetric":0,"t_first_token":350490.972102682,"t_first_token_unix":1779781355.9860466,"t_done":350493.143579389,"t_done_unix":1779781358.1575232},{"request_id":"1340290:18:1878955:1117","session_id":"1340290","input_length":40583,"t_proxy_recv":350498.098186277,"t_decision_unix":1779781363.1121273,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":40448,"affinity_cache_ratio":0.9966734839711209,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":40448,"estimated_new_tokens":135,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1828,"cache_hit":0,"new_prefill":40583,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112926,"ongoing_decode_tokens":112926,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":40583,"score_linear":112926.0,"score_lmetric":40583},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2428,"cache_hit":40448,"new_prefill":135,"score_linear":-40448.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2236,"cache_hit":0,"new_prefill":40583,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":40583,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1780,"cache_hit":0,"new_prefill":40583,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":39559,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":40583,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-40448.0,"chosen_score_lmetric":0,"t_first_token":350498.160519917,"t_first_token_unix":1779781363.1744645,"t_done":350498.376204303,"t_done_unix":1779781363.3901477},{"request_id":"1340290:19:1880565:1118","session_id":"1340290","input_length":40676,"t_proxy_recv":350503.37231362,"t_decision_unix":1779781368.3862545,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":40448,"affinity_cache_ratio":0.9943947290785722,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":40448,"estimated_new_tokens":228,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1828,"cache_hit":0,"new_prefill":40676,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112926,"ongoing_decode_tokens":112926,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":40676,"score_linear":112926.0,"score_lmetric":40676},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2428,"cache_hit":40448,"new_prefill":228,"score_linear":-40448.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2236,"cache_hit":0,"new_prefill":40676,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":40676,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1780,"cache_hit":0,"new_prefill":40676,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":39652,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":40676,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-40448.0,"chosen_score_lmetric":0,"t_first_token":350503.472042237,"t_first_token_unix":1779781368.4859855,"t_done":350504.037044539,"t_done_unix":1779781369.0509884},{"request_id":"1340290:20:1882251:1120","session_id":"1340290","input_length":40883,"t_proxy_recv":350509.184501258,"t_decision_unix":1779781374.1984417,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":40448,"affinity_cache_ratio":0.9893598806349827,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":40448,"estimated_new_tokens":435,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1828,"cache_hit":0,"new_prefill":40883,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112926,"ongoing_decode_tokens":112926,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":40883,"score_linear":112926.0,"score_lmetric":40883},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2428,"cache_hit":40448,"new_prefill":435,"score_linear":-40448.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":119030,"ongoing_decode_tokens":119030,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2236,"cache_hit":0,"new_prefill":40883,"score_linear":119030.0,"score_lmetric":40883},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":40883,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1780,"cache_hit":0,"new_prefill":40883,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":39859,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":40883,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-40448.0,"chosen_score_lmetric":0,"t_first_token":350509.290272918,"t_first_token_unix":1779781374.3042161,"t_done":350509.621108917,"t_done_unix":1779781374.635052},{"request_id":"1270606:12:1881189:1119","session_id":"1270606","input_length":119030,"t_proxy_recv":350505.779643584,"t_decision_unix":1779781370.7935848,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":118784,"affinity_cache_ratio":0.9979332941275308,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":118784,"estimated_new_tokens":246,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1828,"cache_hit":0,"new_prefill":119030,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112926,"ongoing_decode_tokens":112926,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":119030,"score_linear":112926.0,"score_lmetric":119030},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2428,"cache_hit":0,"new_prefill":119030,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2236,"cache_hit":118784,"new_prefill":246,"score_linear":-118784.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":1536,"new_prefill":117494,"score_linear":-1536.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1780,"cache_hit":0,"new_prefill":119030,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":119030,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":119030,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-118784.0,"chosen_score_lmetric":0,"t_first_token":350506.143695659,"t_first_token_unix":1779781371.1576397,"t_done":350510.18571472,"t_done_unix":1779781375.1996584},{"request_id":"1340290:21:1883855:1121","session_id":"1340290","input_length":41002,"t_proxy_recv":350514.650060966,"t_decision_unix":1779781379.664002,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":40448,"affinity_cache_ratio":0.986488463977367,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":40448,"estimated_new_tokens":554,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1828,"cache_hit":0,"new_prefill":41002,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112926,"ongoing_decode_tokens":112926,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":41002,"score_linear":112926.0,"score_lmetric":41002},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2428,"cache_hit":40448,"new_prefill":554,"score_linear":-40448.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2237,"cache_hit":0,"new_prefill":41002,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":41002,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1780,"cache_hit":0,"new_prefill":41002,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":39978,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":41002,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-40448.0,"chosen_score_lmetric":0,"t_first_token":350514.754073971,"t_first_token_unix":1779781379.768018,"t_done":350515.008591068,"t_done_unix":1779781380.0225348},{"request_id":"1340290:22:1884847:1122","session_id":"1340290","input_length":41909,"t_proxy_recv":350518.122650629,"t_decision_unix":1779781383.136592,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":40960,"affinity_cache_ratio":0.9773556992531437,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":40960,"estimated_new_tokens":949,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1828,"cache_hit":0,"new_prefill":41909,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112926,"ongoing_decode_tokens":112926,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":41909,"score_linear":112926.0,"score_lmetric":41909},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2429,"cache_hit":40960,"new_prefill":949,"score_linear":-40960.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2237,"cache_hit":0,"new_prefill":41909,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":41909,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1780,"cache_hit":0,"new_prefill":41909,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":40885,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":41909,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-40960.0,"chosen_score_lmetric":0,"t_first_token":350518.451830198,"t_first_token_unix":1779781383.4657743,"t_done":350518.79476817,"t_done_unix":1779781383.8087118},{"request_id":"1253804:55:1869941:1112","session_id":"1253804","input_length":112926,"t_proxy_recv":350467.16054294,"t_decision_unix":1779781332.1744843,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":1,"chosen_idx":1,"affinity_cache_hit":112640,"affinity_cache_ratio":0.9974673680109098,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":112640,"estimated_new_tokens":286,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8001","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1827,"cache_hit":0,"new_prefill":112926,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":112640,"new_prefill":286,"score_linear":-112640.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2427,"cache_hit":0,"new_prefill":112926,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2234,"cache_hit":3584,"new_prefill":109342,"score_linear":-3584.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":112926,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1780,"cache_hit":3584,"new_prefill":109342,"score_linear":-3584.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":112926,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":112926,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-112640.0,"chosen_score_lmetric":0,"t_first_token":350467.29847144,"t_first_token_unix":1779781332.3124146,"t_done":350520.386994819,"t_done_unix":1779781385.4009383},{"request_id":"1340290:23:1886013:1124","session_id":"1340290","input_length":42086,"t_proxy_recv":350522.152395927,"t_decision_unix":1779781387.166337,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":41472,"affinity_cache_ratio":0.9854108254526446,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":41472,"estimated_new_tokens":614,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1828,"cache_hit":0,"new_prefill":42086,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":42086,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2430,"cache_hit":41472,"new_prefill":614,"score_linear":-41472.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2237,"cache_hit":0,"new_prefill":42086,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":42086,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":63845,"ongoing_decode_tokens":0,"pending_prefill_tokens":5477,"num_requests":1,"active_p_offloads":0,"cached_blocks":1780,"cache_hit":0,"new_prefill":42086,"score_linear":63845.0,"score_lmetric":47563},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":41062,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":42086,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-41472.0,"chosen_score_lmetric":0,"t_first_token":350522.259633392,"t_first_token_unix":1779781387.273577,"t_done":350522.560188144,"t_done_unix":1779781387.5741313},{"request_id":"1270606:12:1886765:1125","session_id":"1270606","input_length":119440,"t_proxy_recv":350524.896242272,"t_decision_unix":1779781389.9101834,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":119296,"affinity_cache_ratio":0.9987943737441393,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":119296,"estimated_new_tokens":144,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1828,"cache_hit":0,"new_prefill":119440,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":119440,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2431,"cache_hit":0,"new_prefill":119440,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2237,"cache_hit":119296,"new_prefill":144,"score_linear":-119296.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":1536,"new_prefill":117904,"score_linear":-1536.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":63845,"ongoing_decode_tokens":63845,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1780,"cache_hit":0,"new_prefill":119440,"score_linear":63845.0,"score_lmetric":119440},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":119440,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":119440,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-119296.0,"chosen_score_lmetric":0,"t_first_token":350525.131097575,"t_first_token_unix":1779781390.1450417,"t_done":350526.187126195,"t_done_unix":1779781391.2010694},{"request_id":"1340290:24:1887125:1126","session_id":"1340290","input_length":42199,"t_proxy_recv":350525.932567506,"t_decision_unix":1779781390.9465091,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":41984,"affinity_cache_ratio":0.9949050925377378,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":41984,"estimated_new_tokens":215,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1828,"cache_hit":0,"new_prefill":42199,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":42199,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2431,"cache_hit":41984,"new_prefill":215,"score_linear":-41984.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":119440,"ongoing_decode_tokens":119440,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2237,"cache_hit":0,"new_prefill":42199,"score_linear":119440.0,"score_lmetric":42199},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":42199,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":63845,"ongoing_decode_tokens":63845,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1780,"cache_hit":0,"new_prefill":42199,"score_linear":63845.0,"score_lmetric":42199},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":41175,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":42199,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-41984.0,"chosen_score_lmetric":0,"t_first_token":350526.036387905,"t_first_token_unix":1779781391.0503309,"t_done":350526.209791455,"t_done_unix":1779781391.2237346},{"request_id":"1340290:25:1888447:1127","session_id":"1340290","input_length":42444,"t_proxy_recv":350530.323846253,"t_decision_unix":1779781395.3377876,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":41984,"affinity_cache_ratio":0.9891621901800018,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":41984,"estimated_new_tokens":460,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1828,"cache_hit":0,"new_prefill":42444,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":42444,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2431,"cache_hit":41984,"new_prefill":460,"score_linear":-41984.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2237,"cache_hit":0,"new_prefill":42444,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":42444,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":63845,"ongoing_decode_tokens":63845,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1780,"cache_hit":0,"new_prefill":42444,"score_linear":63845.0,"score_lmetric":42444},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":41420,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":42444,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-41984.0,"chosen_score_lmetric":0,"t_first_token":350530.472281356,"t_first_token_unix":1779781395.4862247,"t_done":350530.780490345,"t_done_unix":1779781395.7944334},{"request_id":"1277428:20:1885419:1123","session_id":"1277428","input_length":63845,"t_proxy_recv":350520.038458091,"t_decision_unix":1779781385.0523994,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":58368,"affinity_cache_ratio":0.9142141123032343,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":58368,"estimated_new_tokens":5477,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1828,"cache_hit":0,"new_prefill":63845,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":112926,"ongoing_decode_tokens":112926,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":4608,"new_prefill":59237,"score_linear":108318.0,"score_lmetric":59237},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2430,"cache_hit":0,"new_prefill":63845,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2237,"cache_hit":17408,"new_prefill":46437,"score_linear":-17408.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":63845,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1780,"cache_hit":58368,"new_prefill":5477,"score_linear":-58368.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":63845,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":63845,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-58368.0,"chosen_score_lmetric":0,"t_first_token":350522.241842118,"t_first_token_unix":1779781387.2557857,"t_done":350532.409265199,"t_done_unix":1779781397.4232085},{"request_id":"1340290:26:1889402:1128","session_id":"1340290","input_length":48504,"t_proxy_recv":350533.646461242,"t_decision_unix":1779781398.6604028,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":41984,"affinity_cache_ratio":0.8655780966518225,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":41984,"estimated_new_tokens":6520,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1828,"cache_hit":0,"new_prefill":48504,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":48504,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2431,"cache_hit":41984,"new_prefill":6520,"score_linear":-41984.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2237,"cache_hit":0,"new_prefill":48504,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":48504,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1791,"cache_hit":0,"new_prefill":48504,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":47480,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":48504,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-41984.0,"chosen_score_lmetric":0,"t_first_token":350535.542715625,"t_first_token_unix":1779781400.5566592,"t_done":350535.755478776,"t_done_unix":1779781400.7694225},{"request_id":"1270606:13:1890252:1129","session_id":"1270606","input_length":119686,"t_proxy_recv":350536.624267732,"t_decision_unix":1779781401.6382093,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":119296,"affinity_cache_ratio":0.9967414735223836,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":119296,"estimated_new_tokens":390,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1828,"cache_hit":0,"new_prefill":119686,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":119686,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2443,"cache_hit":0,"new_prefill":119686,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2237,"cache_hit":119296,"new_prefill":390,"score_linear":-119296.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":1536,"new_prefill":118150,"score_linear":-1536.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1791,"cache_hit":0,"new_prefill":119686,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":119686,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":119686,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-119296.0,"chosen_score_lmetric":0,"t_first_token":350536.99244848,"t_first_token_unix":1779781402.0063922,"t_done":350537.928488185,"t_done_unix":1779781402.9424443},{"request_id":"1340290:27:1892591:1131","session_id":"1340290","input_length":49361,"t_proxy_recv":350544.228438373,"t_decision_unix":1779781409.2423797,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":48128,"affinity_cache_ratio":0.9750207653815766,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":48128,"estimated_new_tokens":1233,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1828,"cache_hit":0,"new_prefill":49361,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":49361,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2443,"cache_hit":48128,"new_prefill":1233,"score_linear":-48128.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":119828,"ongoing_decode_tokens":119828,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2237,"cache_hit":0,"new_prefill":49361,"score_linear":119828.0,"score_lmetric":49361},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":49361,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1791,"cache_hit":0,"new_prefill":49361,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":48337,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":49361,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-48128.0,"chosen_score_lmetric":0,"t_first_token":350544.599659864,"t_first_token_unix":1779781409.6136034,"t_done":350544.928238255,"t_done_unix":1779781409.9421816},{"request_id":"1270606:14:1892412:1130","session_id":"1270606","input_length":119828,"t_proxy_recv":350543.572455117,"t_decision_unix":1779781408.5863965,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":119296,"affinity_cache_ratio":0.9955603031011115,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":119296,"estimated_new_tokens":532,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1828,"cache_hit":0,"new_prefill":119828,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":119828,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2443,"cache_hit":0,"new_prefill":119828,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2237,"cache_hit":119296,"new_prefill":532,"score_linear":-119296.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":1536,"new_prefill":118292,"score_linear":-1536.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1791,"cache_hit":0,"new_prefill":119828,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":119828,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":119828,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-119296.0,"chosen_score_lmetric":0,"t_first_token":350543.915678006,"t_first_token_unix":1779781408.9296215,"t_done":350545.892626736,"t_done_unix":1779781410.9065702},{"request_id":"1340290:28:1893837:1132","session_id":"1340290","input_length":49454,"t_proxy_recv":350548.539910455,"t_decision_unix":1779781413.5538516,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":49152,"affinity_cache_ratio":0.9938933149997978,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":49152,"estimated_new_tokens":302,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1828,"cache_hit":0,"new_prefill":49454,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":49454,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2445,"cache_hit":49152,"new_prefill":302,"score_linear":-49152.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2238,"cache_hit":0,"new_prefill":49454,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":49454,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1791,"cache_hit":0,"new_prefill":49454,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":48430,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":49454,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-49152.0,"chosen_score_lmetric":0,"t_first_token":350548.646302284,"t_first_token_unix":1779781413.6602457,"t_done":350549.233513363,"t_done_unix":1779781414.2474568},{"request_id":"1305906:9:1893880:1133","session_id":"1305906","input_length":59265,"t_proxy_recv":350548.706512259,"t_decision_unix":1779781413.7204537,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":58880,"affinity_cache_ratio":0.9935037543237999,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":58880,"estimated_new_tokens":385,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1828,"cache_hit":0,"new_prefill":59265,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":59265,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":49454,"ongoing_decode_tokens":49454,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2445,"cache_hit":58880,"new_prefill":385,"score_linear":-9426.0,"score_lmetric":385},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2238,"cache_hit":0,"new_prefill":59265,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":59265,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1791,"cache_hit":0,"new_prefill":59265,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":59265,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":59265,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-9426.0,"chosen_score_lmetric":385,"t_first_token":350549.014509423,"t_first_token_unix":1779781414.0284534,"t_done":350549.71346039,"t_done_unix":1779781414.7274034},{"request_id":"1270606:15:1895310:1134","session_id":"1270606","input_length":120058,"t_proxy_recv":350553.400975615,"t_decision_unix":1779781418.4149165,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":119808,"affinity_cache_ratio":0.9979176731246564,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":119808,"estimated_new_tokens":250,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1828,"cache_hit":0,"new_prefill":120058,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":120058,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2445,"cache_hit":0,"new_prefill":120058,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2238,"cache_hit":119808,"new_prefill":250,"score_linear":-119808.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":1536,"new_prefill":118522,"score_linear":-1536.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1791,"cache_hit":0,"new_prefill":120058,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":120058,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":120058,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-119808.0,"chosen_score_lmetric":0,"t_first_token":350553.647987363,"t_first_token_unix":1779781418.6619308,"t_done":350554.639581549,"t_done_unix":1779781419.6535254},{"request_id":"1340290:29:1896119:1136","session_id":"1340290","input_length":49576,"t_proxy_recv":350555.862344748,"t_decision_unix":1779781420.876286,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":49152,"affinity_cache_ratio":0.9914474745844764,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":49152,"estimated_new_tokens":424,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":135905,"ongoing_decode_tokens":135905,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1828,"cache_hit":0,"new_prefill":49576,"score_linear":135905.0,"score_lmetric":49576},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":49576,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2445,"cache_hit":49152,"new_prefill":424,"score_linear":-49152.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2238,"cache_hit":0,"new_prefill":49576,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":49576,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1791,"cache_hit":0,"new_prefill":49576,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":48552,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":49576,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-49152.0,"chosen_score_lmetric":0,"t_first_token":350555.966739478,"t_first_token_unix":1779781420.980683,"t_done":350556.218551111,"t_done_unix":1779781421.2324944},{"request_id":"1356885:13:1895311:1135","session_id":"1356885","input_length":135905,"t_proxy_recv":350553.41912525,"t_decision_unix":1779781418.433067,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":135168,"affinity_cache_ratio":0.9945770942938081,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":135168,"estimated_new_tokens":737,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1828,"cache_hit":135168,"new_prefill":737,"score_linear":-135168.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":1024,"new_prefill":134881,"score_linear":-1024.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2445,"cache_hit":0,"new_prefill":135905,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":120058,"ongoing_decode_tokens":0,"pending_prefill_tokens":250,"num_requests":1,"active_p_offloads":0,"cached_blocks":2238,"cache_hit":0,"new_prefill":135905,"score_linear":120058.0,"score_lmetric":136155},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":6656,"new_prefill":129249,"score_linear":-6656.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1791,"cache_hit":0,"new_prefill":135905,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":135905,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":135905,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-135168.0,"chosen_score_lmetric":0,"t_first_token":350553.84583412,"t_first_token_unix":1779781418.8597775,"t_done":350556.289326491,"t_done_unix":1779781421.3032696},{"request_id":"1340290:30:1896991:1137","session_id":"1340290","input_length":49695,"t_proxy_recv":350558.835666988,"t_decision_unix":1779781423.849608,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":49152,"affinity_cache_ratio":0.9890733474192575,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":49152,"estimated_new_tokens":543,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1829,"cache_hit":0,"new_prefill":49695,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":49695,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2445,"cache_hit":49152,"new_prefill":543,"score_linear":-49152.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2238,"cache_hit":0,"new_prefill":49695,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":49695,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1791,"cache_hit":0,"new_prefill":49695,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":48671,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":49695,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-49152.0,"chosen_score_lmetric":0,"t_first_token":350558.939943953,"t_first_token_unix":1779781423.9538872,"t_done":350560.459622976,"t_done_unix":1779781425.4735663},{"request_id":"1340290:31:1899492:1138","session_id":"1340290","input_length":49927,"t_proxy_recv":350567.219067763,"t_decision_unix":1779781432.2330089,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":49664,"affinity_cache_ratio":0.9947323091713902,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":49664,"estimated_new_tokens":263,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1829,"cache_hit":0,"new_prefill":49927,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":49927,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2446,"cache_hit":49664,"new_prefill":263,"score_linear":-49664.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2238,"cache_hit":0,"new_prefill":49927,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":49927,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1791,"cache_hit":0,"new_prefill":49927,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":48903,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":49927,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-49664.0,"chosen_score_lmetric":0,"t_first_token":350567.308167194,"t_first_token_unix":1779781432.3221102,"t_done":350567.973167784,"t_done_unix":1779781432.9871109},{"request_id":"1270606:16:1902252:1140","session_id":"1270606","input_length":120572,"t_proxy_recv":350575.729123665,"t_decision_unix":1779781440.7430649,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":119808,"affinity_cache_ratio":0.9936635371396344,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":119808,"estimated_new_tokens":764,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1829,"cache_hit":0,"new_prefill":120572,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":120572,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":50078,"ongoing_decode_tokens":50078,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2446,"cache_hit":0,"new_prefill":120572,"score_linear":50078.0,"score_lmetric":120572},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2238,"cache_hit":119808,"new_prefill":764,"score_linear":-119808.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":1536,"new_prefill":119036,"score_linear":-1536.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1791,"cache_hit":0,"new_prefill":120572,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":120572,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":120572,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-119808.0,"chosen_score_lmetric":0,"t_first_token":350576.306796657,"t_first_token_unix":1779781441.3207397,"t_done":350577.274059307,"t_done_unix":1779781442.2880025},{"request_id":"1270606:17:1908800:1141","session_id":"1270606","input_length":121006,"t_proxy_recv":350597.316766528,"t_decision_unix":1779781462.3307078,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":120320,"affinity_cache_ratio":0.9943308596267953,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":120320,"estimated_new_tokens":686,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1829,"cache_hit":0,"new_prefill":121006,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":121006,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":50078,"ongoing_decode_tokens":50078,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2446,"cache_hit":0,"new_prefill":121006,"score_linear":50078.0,"score_lmetric":121006},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2239,"cache_hit":120320,"new_prefill":686,"score_linear":-120320.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":1536,"new_prefill":119470,"score_linear":-1536.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1791,"cache_hit":0,"new_prefill":121006,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":121006,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":121006,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-120320.0,"chosen_score_lmetric":0,"t_first_token":350597.895439596,"t_first_token_unix":1779781462.9093835,"t_done":350598.521622491,"t_done_unix":1779781463.535566},{"request_id":"1340290:32:1901260:1139","session_id":"1340290","input_length":50078,"t_proxy_recv":350572.629912153,"t_decision_unix":1779781437.6438537,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":49664,"affinity_cache_ratio":0.9917328966811774,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":49664,"estimated_new_tokens":414,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1829,"cache_hit":0,"new_prefill":50078,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":50078,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2446,"cache_hit":49664,"new_prefill":414,"score_linear":-49664.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2238,"cache_hit":0,"new_prefill":50078,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":50078,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1791,"cache_hit":0,"new_prefill":50078,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":49054,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":50078,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-49664.0,"chosen_score_lmetric":0,"t_first_token":350572.746551578,"t_first_token_unix":1779781437.760495,"t_done":350603.573849267,"t_done_unix":1779781468.5877924},{"request_id":"1270606:18:1915824:1142","session_id":"1270606","input_length":121288,"t_proxy_recv":350620.297880285,"t_decision_unix":1779781485.3118215,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":120832,"affinity_cache_ratio":0.9962403535386848,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":120832,"estimated_new_tokens":456,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1829,"cache_hit":0,"new_prefill":121288,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":121288,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2448,"cache_hit":0,"new_prefill":121288,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2240,"cache_hit":120832,"new_prefill":456,"score_linear":-120832.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":1536,"new_prefill":119752,"score_linear":-1536.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1791,"cache_hit":0,"new_prefill":121288,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":121288,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":121288,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-120832.0,"chosen_score_lmetric":0,"t_first_token":350620.66503368,"t_first_token_unix":1779781485.678977,"t_done":350621.546728717,"t_done_unix":1779781486.560672},{"request_id":"1270606:19:1920499:1143","session_id":"1270606","input_length":121560,"t_proxy_recv":350635.435269874,"t_decision_unix":1779781500.449211,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":121344,"affinity_cache_ratio":0.9982230997038499,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":121344,"estimated_new_tokens":216,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1829,"cache_hit":0,"new_prefill":121560,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":121560,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2448,"cache_hit":0,"new_prefill":121560,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2241,"cache_hit":121344,"new_prefill":216,"score_linear":-121344.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":1536,"new_prefill":120024,"score_linear":-1536.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1791,"cache_hit":0,"new_prefill":121560,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":121560,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":121560,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-121344.0,"chosen_score_lmetric":0,"t_first_token":350635.795986839,"t_first_token_unix":1779781500.8099303,"t_done":350636.785930889,"t_done_unix":1779781501.7998745},{"request_id":"1270606:20:1922639:1144","session_id":"1270606","input_length":121704,"t_proxy_recv":350642.891518549,"t_decision_unix":1779781507.9054596,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":121344,"affinity_cache_ratio":0.9970420035495957,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":121344,"estimated_new_tokens":360,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1829,"cache_hit":0,"new_prefill":121704,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":121704,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2448,"cache_hit":0,"new_prefill":121704,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2241,"cache_hit":121344,"new_prefill":360,"score_linear":-121344.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":1536,"new_prefill":120168,"score_linear":-1536.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1791,"cache_hit":0,"new_prefill":121704,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":121704,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":121704,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-121344.0,"chosen_score_lmetric":0,"t_first_token":350643.113752894,"t_first_token_unix":1779781508.1276963,"t_done":350644.445860056,"t_done_unix":1779781509.459803},{"request_id":"1305906:10:1928316:1145","session_id":"1305906","input_length":61801,"t_proxy_recv":350661.434988105,"t_decision_unix":1779781526.448929,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":58880,"affinity_cache_ratio":0.9527353926311872,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":58880,"estimated_new_tokens":2921,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1829,"cache_hit":0,"new_prefill":61801,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":61801,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2448,"cache_hit":58880,"new_prefill":2921,"score_linear":-58880.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2241,"cache_hit":0,"new_prefill":61801,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":61801,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1791,"cache_hit":0,"new_prefill":61801,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":61801,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":61801,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-58880.0,"chosen_score_lmetric":0,"t_first_token":350662.470231186,"t_first_token_unix":1779781527.4841752,"t_done":350662.82981376,"t_done_unix":1779781527.8437572},{"request_id":"1340290:33:1939781:1146","session_id":"1340290","input_length":53990,"t_proxy_recv":350699.174810386,"t_decision_unix":1779781564.1887515,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":53760,"affinity_cache_ratio":0.9957399518429338,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":53760,"estimated_new_tokens":230,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1829,"cache_hit":0,"new_prefill":53990,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":53990,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2453,"cache_hit":53760,"new_prefill":230,"score_linear":-53760.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2241,"cache_hit":0,"new_prefill":53990,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":53990,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1791,"cache_hit":0,"new_prefill":53990,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":52966,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":53990,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-53760.0,"chosen_score_lmetric":0,"t_first_token":350699.268104728,"t_first_token_unix":1779781564.2820477,"t_done":350700.372081114,"t_done_unix":1779781565.3860247},{"request_id":"1340290:34:1942249:1147","session_id":"1340290","input_length":54164,"t_proxy_recv":350707.052387139,"t_decision_unix":1779781572.066328,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":53760,"affinity_cache_ratio":0.992541171257662,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":53760,"estimated_new_tokens":404,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1829,"cache_hit":0,"new_prefill":54164,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":54164,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2453,"cache_hit":53760,"new_prefill":404,"score_linear":-53760.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2241,"cache_hit":0,"new_prefill":54164,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":54164,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1791,"cache_hit":0,"new_prefill":54164,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":53140,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":54164,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-53760.0,"chosen_score_lmetric":0,"t_first_token":350707.14542489,"t_first_token_unix":1779781572.1593685,"t_done":350709.634935619,"t_done_unix":1779781574.648879},{"request_id":"1356885:14:1948151:1148","session_id":"1356885","input_length":136210,"t_proxy_recv":350726.804214274,"t_decision_unix":1779781591.8181558,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":135680,"affinity_cache_ratio":0.9961089494163424,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":135680,"estimated_new_tokens":530,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1829,"cache_hit":135680,"new_prefill":530,"score_linear":-135680.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":1024,"new_prefill":135186,"score_linear":-1024.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2454,"cache_hit":0,"new_prefill":136210,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2241,"cache_hit":0,"new_prefill":136210,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":6656,"new_prefill":129554,"score_linear":-6656.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1791,"cache_hit":0,"new_prefill":136210,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":136210,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":136210,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-135680.0,"chosen_score_lmetric":0,"t_first_token":350727.132818496,"t_first_token_unix":1779781592.146762,"t_done":350728.843574619,"t_done_unix":1779781593.8575187},{"request_id":"1340290:35:1964727:1149","session_id":"1340290","input_length":54498,"t_proxy_recv":350781.533161896,"t_decision_unix":1779781646.5471034,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":54272,"affinity_cache_ratio":0.9958530588278469,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":54272,"estimated_new_tokens":226,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1830,"cache_hit":0,"new_prefill":54498,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":54498,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2454,"cache_hit":54272,"new_prefill":226,"score_linear":-54272.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2241,"cache_hit":0,"new_prefill":54498,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":54498,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1791,"cache_hit":0,"new_prefill":54498,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":53474,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":54498,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-54272.0,"chosen_score_lmetric":0,"t_first_token":350781.627898892,"t_first_token_unix":1779781646.6418421,"t_done":350785.448366817,"t_done_unix":1779781650.4623103},{"request_id":"1340290:36:1969509:1150","session_id":"1340290","input_length":54811,"t_proxy_recv":350797.273151231,"t_decision_unix":1779781662.2870927,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":54784,"affinity_cache_ratio":0.9995073981500063,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":54784,"estimated_new_tokens":27,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1830,"cache_hit":0,"new_prefill":54811,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":54811,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2455,"cache_hit":54784,"new_prefill":27,"score_linear":-54784.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2241,"cache_hit":0,"new_prefill":54811,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":54811,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1791,"cache_hit":0,"new_prefill":54811,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":53787,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":54811,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-54784.0,"chosen_score_lmetric":0,"t_first_token":350797.349986136,"t_first_token_unix":1779781662.3639293,"t_done":350797.810278875,"t_done_unix":1779781662.8242218},{"request_id":"1340290:37:1971106:1151","session_id":"1340290","input_length":55358,"t_proxy_recv":350802.428417178,"t_decision_unix":1779781667.4423585,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":54784,"affinity_cache_ratio":0.9896311282922071,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":54784,"estimated_new_tokens":574,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1830,"cache_hit":0,"new_prefill":55358,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":55358,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2455,"cache_hit":54784,"new_prefill":574,"score_linear":-54784.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2241,"cache_hit":0,"new_prefill":55358,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":55358,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1791,"cache_hit":0,"new_prefill":55358,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":54334,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":55358,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-54784.0,"chosen_score_lmetric":0,"t_first_token":350802.722551433,"t_first_token_unix":1779781667.7364945,"t_done":350803.190378299,"t_done_unix":1779781668.2043214},{"request_id":"1340290:38:1972289:1152","session_id":"1340290","input_length":56006,"t_proxy_recv":350806.356952121,"t_decision_unix":1779781671.3708935,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":55296,"affinity_cache_ratio":0.9873227868442667,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":55296,"estimated_new_tokens":710,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1830,"cache_hit":0,"new_prefill":56006,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":56006,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2456,"cache_hit":55296,"new_prefill":710,"score_linear":-55296.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2241,"cache_hit":0,"new_prefill":56006,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":56006,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1791,"cache_hit":0,"new_prefill":56006,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":54982,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":56006,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-55296.0,"chosen_score_lmetric":0,"t_first_token":350806.660451723,"t_first_token_unix":1779781671.674395,"t_done":350807.332027733,"t_done_unix":1779781672.3459709},{"request_id":"1340290:39:1973889:1153","session_id":"1340290","input_length":56242,"t_proxy_recv":350811.783115341,"t_decision_unix":1779781676.7970562,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":55808,"affinity_cache_ratio":0.9922833469649017,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":55808,"estimated_new_tokens":434,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1830,"cache_hit":0,"new_prefill":56242,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":56242,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2457,"cache_hit":55808,"new_prefill":434,"score_linear":-55808.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2241,"cache_hit":0,"new_prefill":56242,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":56242,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1791,"cache_hit":0,"new_prefill":56242,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":55218,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":56242,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-55808.0,"chosen_score_lmetric":0,"t_first_token":350811.967123642,"t_first_token_unix":1779781676.9810667,"t_done":350812.63864431,"t_done_unix":1779781677.6525874},{"request_id":"1340290:40:1975278:1154","session_id":"1340290","input_length":56416,"t_proxy_recv":350816.344747709,"t_decision_unix":1779781681.358689,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":56320,"affinity_cache_ratio":0.998298355076574,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":56320,"estimated_new_tokens":96,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1830,"cache_hit":0,"new_prefill":56416,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":56416,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2458,"cache_hit":56320,"new_prefill":96,"score_linear":-56320.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2241,"cache_hit":0,"new_prefill":56416,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":56416,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1791,"cache_hit":0,"new_prefill":56416,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":55392,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":56416,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-56320.0,"chosen_score_lmetric":0,"t_first_token":350816.45900978,"t_first_token_unix":1779781681.4729533,"t_done":350816.949924247,"t_done_unix":1779781681.963868},{"request_id":"1340290:41:1977055:1156","session_id":"1340290","input_length":57181,"t_proxy_recv":350822.280187727,"t_decision_unix":1779781687.2941294,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":56320,"affinity_cache_ratio":0.9849425508473094,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":56320,"estimated_new_tokens":861,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1830,"cache_hit":0,"new_prefill":57181,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":57181,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2458,"cache_hit":56320,"new_prefill":861,"score_linear":-56320.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2241,"cache_hit":0,"new_prefill":57181,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":57181,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":67793,"ongoing_decode_tokens":0,"pending_prefill_tokens":2769,"num_requests":1,"active_p_offloads":0,"cached_blocks":1791,"cache_hit":0,"new_prefill":57181,"score_linear":67793.0,"score_lmetric":59950},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":56157,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":57181,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-56320.0,"chosen_score_lmetric":0,"t_first_token":350822.688562137,"t_first_token_unix":1779781687.7025058,"t_done":350824.690242556,"t_done_unix":1779781689.704186},{"request_id":"1277428:21:1976968:1155","session_id":"1277428","input_length":67793,"t_proxy_recv":350821.998306879,"t_decision_unix":1779781687.0122445,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":65024,"affinity_cache_ratio":0.9591550750077442,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":65024,"estimated_new_tokens":2769,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1830,"cache_hit":0,"new_prefill":67793,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":4608,"new_prefill":63185,"score_linear":-4608.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2458,"cache_hit":0,"new_prefill":67793,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2241,"cache_hit":17408,"new_prefill":50385,"score_linear":-17408.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":67793,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1791,"cache_hit":65024,"new_prefill":2769,"score_linear":-65024.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":67793,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":67793,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-65024.0,"chosen_score_lmetric":0,"t_first_token":350823.245587639,"t_first_token_unix":1779781688.2595317,"t_done":350829.151172379,"t_done_unix":1779781694.1651156},{"request_id":"1340290:42:1982198:1157","session_id":"1340290","input_length":57463,"t_proxy_recv":350839.186767839,"t_decision_unix":1779781704.2007089,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":57344,"affinity_cache_ratio":0.9979291022048971,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":57344,"estimated_new_tokens":119,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1830,"cache_hit":0,"new_prefill":57463,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":57463,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2460,"cache_hit":57344,"new_prefill":119,"score_linear":-57344.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2241,"cache_hit":0,"new_prefill":57463,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":57463,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1797,"cache_hit":0,"new_prefill":57463,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":56439,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":57463,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-57344.0,"chosen_score_lmetric":0,"t_first_token":350839.280655501,"t_first_token_unix":1779781704.2945988,"t_done":350839.780796758,"t_done_unix":1779781704.7947402},{"request_id":"1340290:43:1983901:1158","session_id":"1340290","input_length":57640,"t_proxy_recv":350844.580907927,"t_decision_unix":1779781709.5948493,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":57344,"affinity_cache_ratio":0.9948646773074254,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":57344,"estimated_new_tokens":296,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1830,"cache_hit":0,"new_prefill":57640,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":57640,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2460,"cache_hit":57344,"new_prefill":296,"score_linear":-57344.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2241,"cache_hit":0,"new_prefill":57640,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":57640,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1797,"cache_hit":0,"new_prefill":57640,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":56616,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":57640,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-57344.0,"chosen_score_lmetric":0,"t_first_token":350844.735379844,"t_first_token_unix":1779781709.7493234,"t_done":350845.258215895,"t_done_unix":1779781710.2721596},{"request_id":"1340290:44:1986230:1159","session_id":"1340290","input_length":57809,"t_proxy_recv":350852.663430716,"t_decision_unix":1779781717.677372,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":57344,"affinity_cache_ratio":0.9919562697849815,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":57344,"estimated_new_tokens":465,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1830,"cache_hit":0,"new_prefill":57809,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":57809,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2460,"cache_hit":57344,"new_prefill":465,"score_linear":-57344.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2241,"cache_hit":0,"new_prefill":57809,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":57809,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1797,"cache_hit":0,"new_prefill":57809,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":56785,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":57809,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-57344.0,"chosen_score_lmetric":0,"t_first_token":350852.814764284,"t_first_token_unix":1779781717.828708,"t_done":350853.539520427,"t_done_unix":1779781718.5534637},{"request_id":"1277428:21:1989132:1160","session_id":"1277428","input_length":69987,"t_proxy_recv":350862.082950821,"t_decision_unix":1779781727.0968924,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":68096,"affinity_cache_ratio":0.9729806964150485,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":68096,"estimated_new_tokens":1891,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1830,"cache_hit":0,"new_prefill":69987,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":4608,"new_prefill":65379,"score_linear":-4608.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2461,"cache_hit":0,"new_prefill":69987,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2241,"cache_hit":17408,"new_prefill":52579,"score_linear":-17408.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":69987,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1797,"cache_hit":68096,"new_prefill":1891,"score_linear":-68096.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":69987,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":69987,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-68096.0,"chosen_score_lmetric":0,"t_first_token":350862.849953319,"t_first_token_unix":1779781727.8638973,"t_done":350863.20114898,"t_done_unix":1779781728.2150922},{"request_id":"1277428:22:1990436:1161","session_id":"1277428","input_length":70055,"t_proxy_recv":350866.621629169,"t_decision_unix":1779781731.6355705,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":69632,"affinity_cache_ratio":0.993961887088716,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":69632,"estimated_new_tokens":423,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1830,"cache_hit":0,"new_prefill":70055,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":4608,"new_prefill":65447,"score_linear":-4608.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2461,"cache_hit":0,"new_prefill":70055,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2241,"cache_hit":17408,"new_prefill":52647,"score_linear":-17408.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":70055,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":69632,"new_prefill":423,"score_linear":-69632.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":70055,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":70055,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-69632.0,"chosen_score_lmetric":0,"t_first_token":350866.734963714,"t_first_token_unix":1779781731.748907,"t_done":350867.149748031,"t_done_unix":1779781732.1636913},{"request_id":"1340290:45:1991694:1162","session_id":"1340290","input_length":58053,"t_proxy_recv":350870.636293657,"t_decision_unix":1779781735.6502347,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":57856,"affinity_cache_ratio":0.9966065491878111,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":57856,"estimated_new_tokens":197,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1830,"cache_hit":0,"new_prefill":58053,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":58053,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2461,"cache_hit":57856,"new_prefill":197,"score_linear":-57856.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2241,"cache_hit":0,"new_prefill":58053,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":58053,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":58053,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":57029,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":58053,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-57856.0,"chosen_score_lmetric":0,"t_first_token":350870.824981013,"t_first_token_unix":1779781735.8389242,"t_done":350871.08383063,"t_done_unix":1779781736.0977736},{"request_id":"1340290:46:1992980:1163","session_id":"1340290","input_length":58122,"t_proxy_recv":350874.860730522,"t_decision_unix":1779781739.8746715,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":57856,"affinity_cache_ratio":0.9954234197033826,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":57856,"estimated_new_tokens":266,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1830,"cache_hit":0,"new_prefill":58122,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":58122,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2461,"cache_hit":57856,"new_prefill":266,"score_linear":-57856.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2241,"cache_hit":0,"new_prefill":58122,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":58122,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":58122,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":57098,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":58122,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-57856.0,"chosen_score_lmetric":0,"t_first_token":350874.956390731,"t_first_token_unix":1779781739.9703345,"t_done":350875.38096559,"t_done_unix":1779781740.3949087},{"request_id":"1340290:47:1995358:1164","session_id":"1340290","input_length":58307,"t_proxy_recv":350882.7236921,"t_decision_unix":1779781747.7376332,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":57856,"affinity_cache_ratio":0.9922650796645343,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":57856,"estimated_new_tokens":451,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1830,"cache_hit":0,"new_prefill":58307,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":58307,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2461,"cache_hit":57856,"new_prefill":451,"score_linear":-57856.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2241,"cache_hit":0,"new_prefill":58307,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":58307,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":58307,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":57283,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":58307,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-57856.0,"chosen_score_lmetric":0,"t_first_token":350882.861047672,"t_first_token_unix":1779781747.8749912,"t_done":350883.118065653,"t_done_unix":1779781748.1320088},{"request_id":"1340290:48:1996945:1165","session_id":"1340290","input_length":58339,"t_proxy_recv":350888.028395296,"t_decision_unix":1779781753.0423367,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":57856,"affinity_cache_ratio":0.9917208042647285,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":57856,"estimated_new_tokens":483,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1830,"cache_hit":0,"new_prefill":58339,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":58339,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2461,"cache_hit":57856,"new_prefill":483,"score_linear":-57856.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2241,"cache_hit":0,"new_prefill":58339,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":58339,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":58339,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":57315,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":58339,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-57856.0,"chosen_score_lmetric":0,"t_first_token":350888.105294447,"t_first_token_unix":1779781753.1192372,"t_done":350890.148050704,"t_done_unix":1779781755.161994},{"request_id":"1356885:15:1998292:1166","session_id":"1356885","input_length":136653,"t_proxy_recv":350892.498243378,"t_decision_unix":1779781757.5121849,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":136192,"affinity_cache_ratio":0.9966264919174844,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":136192,"estimated_new_tokens":461,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1830,"cache_hit":136192,"new_prefill":461,"score_linear":-136192.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":1024,"new_prefill":135629,"score_linear":-1024.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2462,"cache_hit":0,"new_prefill":136653,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2241,"cache_hit":0,"new_prefill":136653,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":6656,"new_prefill":129997,"score_linear":-6656.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":136653,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":136653,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":136653,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-136192.0,"chosen_score_lmetric":0,"t_first_token":350893.155576704,"t_first_token_unix":1779781758.1695201,"t_done":350894.840202467,"t_done_unix":1779781759.8541458},{"request_id":"1340290:48:1999591:1167","session_id":"1340290","input_length":58622,"t_proxy_recv":350897.043523719,"t_decision_unix":1779781762.057465,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":58368,"affinity_cache_ratio":0.9956671556753437,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":58368,"estimated_new_tokens":254,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1831,"cache_hit":0,"new_prefill":58622,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":58622,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2462,"cache_hit":58368,"new_prefill":254,"score_linear":-58368.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2241,"cache_hit":0,"new_prefill":58622,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":58622,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":58622,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":57598,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":58622,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-58368.0,"chosen_score_lmetric":0,"t_first_token":350897.142225157,"t_first_token_unix":1779781762.1561682,"t_done":350897.750382342,"t_done_unix":1779781762.7643259},{"request_id":"1340290:48:2001313:1168","session_id":"1340290","input_length":58808,"t_proxy_recv":350902.510440475,"t_decision_unix":1779781767.5243816,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":58368,"affinity_cache_ratio":0.9925180247585362,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":58368,"estimated_new_tokens":440,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1831,"cache_hit":0,"new_prefill":58808,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":58808,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2462,"cache_hit":58368,"new_prefill":440,"score_linear":-58368.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2241,"cache_hit":0,"new_prefill":58808,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":58808,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":58808,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":57784,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":58808,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-58368.0,"chosen_score_lmetric":0,"t_first_token":350902.664676059,"t_first_token_unix":1779781767.6786194,"t_done":350903.18776262,"t_done_unix":1779781768.201706},{"request_id":"1340290:48:2003195:1169","session_id":"1340290","input_length":58977,"t_proxy_recv":350908.83440969,"t_decision_unix":1779781773.848351,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":58368,"affinity_cache_ratio":0.989673940688743,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":58368,"estimated_new_tokens":609,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1831,"cache_hit":0,"new_prefill":58977,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":58977,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2462,"cache_hit":58368,"new_prefill":609,"score_linear":-58368.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2241,"cache_hit":0,"new_prefill":58977,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":58977,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":58977,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":57953,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":58977,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-58368.0,"chosen_score_lmetric":0,"t_first_token":350908.98781471,"t_first_token_unix":1779781774.0017586,"t_done":350909.885475211,"t_done_unix":1779781774.8994186},{"request_id":"1340290:48:2005449:1170","session_id":"1340290","input_length":59220,"t_proxy_recv":350916.226330912,"t_decision_unix":1779781781.2402718,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":58880,"affinity_cache_ratio":0.994258696386356,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":58880,"estimated_new_tokens":340,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1831,"cache_hit":0,"new_prefill":59220,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":59220,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2463,"cache_hit":58880,"new_prefill":340,"score_linear":-58880.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2241,"cache_hit":0,"new_prefill":59220,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":59220,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":59220,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":58196,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":59220,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-58880.0,"chosen_score_lmetric":0,"t_first_token":350916.367071948,"t_first_token_unix":1779781781.381015,"t_done":350917.222099919,"t_done_unix":1779781782.2360432},{"request_id":"1340290:48:2007287:1171","session_id":"1340290","input_length":59487,"t_proxy_recv":350922.348643514,"t_decision_unix":1779781787.3625848,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":58880,"affinity_cache_ratio":0.9897960899019954,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":58880,"estimated_new_tokens":607,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1831,"cache_hit":0,"new_prefill":59487,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":59487,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2463,"cache_hit":58880,"new_prefill":607,"score_linear":-58880.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2241,"cache_hit":0,"new_prefill":59487,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":59487,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":59487,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":58463,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":59487,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-58880.0,"chosen_score_lmetric":0,"t_first_token":350922.541849169,"t_first_token_unix":1779781787.555793,"t_done":350922.930661302,"t_done_unix":1779781787.9446046},{"request_id":"1340290:48:2008380:1173","session_id":"1340290","input_length":60935,"t_proxy_recv":350926.469894998,"t_decision_unix":1779781791.4838362,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":59392,"affinity_cache_ratio":0.9746779355050463,"affinity_num_requests":1,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":59392,"estimated_new_tokens":1543,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1831,"cache_hit":0,"new_prefill":60935,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":60935,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":66506,"ongoing_decode_tokens":0,"pending_prefill_tokens":5066,"num_requests":1,"active_p_offloads":0,"cached_blocks":2464,"cache_hit":59392,"new_prefill":1543,"score_linear":7114.0,"score_lmetric":6609},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2241,"cache_hit":0,"new_prefill":60935,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":60935,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":60935,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":59911,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":60935,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":7114.0,"chosen_score_lmetric":6609,"t_first_token":350928.016514312,"t_first_token_unix":1779781793.030458,"t_done":350929.043667943,"t_done_unix":1779781794.057611},{"request_id":"1305906:11:2008110:1172","session_id":"1305906","input_length":66506,"t_proxy_recv":350925.440756325,"t_decision_unix":1779781790.4546974,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":61440,"affinity_cache_ratio":0.9238264216762397,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":61440,"estimated_new_tokens":5066,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1831,"cache_hit":0,"new_prefill":66506,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":66506,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2464,"cache_hit":61440,"new_prefill":5066,"score_linear":-61440.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2241,"cache_hit":0,"new_prefill":66506,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":66506,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":66506,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":66506,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":66506,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-61440.0,"chosen_score_lmetric":0,"t_first_token":350927.402639618,"t_first_token_unix":1779781792.416583,"t_done":350930.679234041,"t_done_unix":1779781795.693178},{"request_id":"1340290:48:2010180:1174","session_id":"1340290","input_length":61080,"t_proxy_recv":350932.309712333,"t_decision_unix":1779781797.3236535,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":60928,"affinity_cache_ratio":0.9975114603798297,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":60928,"estimated_new_tokens":152,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1831,"cache_hit":0,"new_prefill":61080,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":61080,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2477,"cache_hit":60928,"new_prefill":152,"score_linear":-60928.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2241,"cache_hit":0,"new_prefill":61080,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":61080,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":61080,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":60056,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":61080,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-60928.0,"chosen_score_lmetric":0,"t_first_token":350932.444052085,"t_first_token_unix":1779781797.4579954,"t_done":350933.483078299,"t_done_unix":1779781798.497024},{"request_id":"1340290:48:2012871:1175","session_id":"1340290","input_length":61365,"t_proxy_recv":350940.988258982,"t_decision_unix":1779781806.0022001,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":60928,"affinity_cache_ratio":0.9928786767701459,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":60928,"estimated_new_tokens":437,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1831,"cache_hit":0,"new_prefill":61365,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":61365,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2477,"cache_hit":60928,"new_prefill":437,"score_linear":-60928.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2241,"cache_hit":0,"new_prefill":61365,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":61365,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":61365,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":60341,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":61365,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-60928.0,"chosen_score_lmetric":0,"t_first_token":350941.185187547,"t_first_token_unix":1779781806.1991308,"t_done":350941.717095495,"t_done_unix":1779781806.7310388},{"request_id":"1340290:48:2014153:1176","session_id":"1340290","input_length":61518,"t_proxy_recv":350945.397665718,"t_decision_unix":1779781810.411607,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":60928,"affinity_cache_ratio":0.9904093110959394,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":60928,"estimated_new_tokens":590,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1831,"cache_hit":0,"new_prefill":61518,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":61518,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2477,"cache_hit":60928,"new_prefill":590,"score_linear":-60928.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2241,"cache_hit":0,"new_prefill":61518,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":61518,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":61518,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":60494,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":61518,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-60928.0,"chosen_score_lmetric":0,"t_first_token":350945.520934362,"t_first_token_unix":1779781810.5348775,"t_done":350946.025473264,"t_done_unix":1779781811.0394166},{"request_id":"1340290:48:2015982:1177","session_id":"1340290","input_length":69113,"t_proxy_recv":350951.480412438,"t_decision_unix":1779781816.494354,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":61440,"affinity_cache_ratio":0.8889789185826111,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":61440,"estimated_new_tokens":7673,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1831,"cache_hit":0,"new_prefill":69113,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":69113,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2478,"cache_hit":61440,"new_prefill":7673,"score_linear":-61440.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2241,"cache_hit":0,"new_prefill":69113,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":69113,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":69113,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":68089,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":69113,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-61440.0,"chosen_score_lmetric":0,"t_first_token":350954.715036477,"t_first_token_unix":1779781819.72898,"t_done":350957.183958805,"t_done_unix":1779781822.197902},{"request_id":"1340290:48:2018973:1178","session_id":"1340290","input_length":69426,"t_proxy_recv":350961.263753158,"t_decision_unix":1779781826.2776945,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":69120,"affinity_cache_ratio":0.9955924293492352,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":69120,"estimated_new_tokens":306,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1831,"cache_hit":0,"new_prefill":69426,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":69426,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2493,"cache_hit":69120,"new_prefill":306,"score_linear":-69120.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2241,"cache_hit":0,"new_prefill":69426,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":69426,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":69426,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":68402,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":69426,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-69120.0,"chosen_score_lmetric":0,"t_first_token":350961.374205722,"t_first_token_unix":1779781826.388149,"t_done":350961.97439121,"t_done_unix":1779781826.9883347},{"request_id":"1340290:48:2020779:1179","session_id":"1340290","input_length":69601,"t_proxy_recv":350967.320045078,"t_decision_unix":1779781832.3339865,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":69120,"affinity_cache_ratio":0.9930891797531645,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":69120,"estimated_new_tokens":481,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1831,"cache_hit":0,"new_prefill":69601,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":69601,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2493,"cache_hit":69120,"new_prefill":481,"score_linear":-69120.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2241,"cache_hit":0,"new_prefill":69601,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":69601,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":69601,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":68577,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":69601,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-69120.0,"chosen_score_lmetric":0,"t_first_token":350967.496600121,"t_first_token_unix":1779781832.5105433,"t_done":350968.076813392,"t_done_unix":1779781833.0907567},{"request_id":"1340290:48:2023753:1180","session_id":"1340290","input_length":69951,"t_proxy_recv":350976.943269075,"t_decision_unix":1779781841.9572103,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":69632,"affinity_cache_ratio":0.9954396649082929,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":69632,"estimated_new_tokens":319,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1831,"cache_hit":0,"new_prefill":69951,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":69951,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2494,"cache_hit":69632,"new_prefill":319,"score_linear":-69632.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2241,"cache_hit":0,"new_prefill":69951,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":69951,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":69951,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":68927,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":69951,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-69632.0,"chosen_score_lmetric":0,"t_first_token":350977.187982765,"t_first_token_unix":1779781842.201926,"t_done":350978.074885038,"t_done_unix":1779781843.0888302},{"request_id":"1340290:48:2026751:1181","session_id":"1340290","input_length":70342,"t_proxy_recv":350987.221014829,"t_decision_unix":1779781852.2349563,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":69632,"affinity_cache_ratio":0.989906457024253,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":69632,"estimated_new_tokens":710,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1831,"cache_hit":0,"new_prefill":70342,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":70342,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2494,"cache_hit":69632,"new_prefill":710,"score_linear":-69632.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2241,"cache_hit":0,"new_prefill":70342,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":70342,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":70342,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":69318,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":70342,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-69632.0,"chosen_score_lmetric":0,"t_first_token":350987.451424928,"t_first_token_unix":1779781852.4653683,"t_done":350987.985026465,"t_done_unix":1779781852.9989698},{"request_id":"1340290:48:2028652:1182","session_id":"1340290","input_length":70503,"t_proxy_recv":350993.474198642,"t_decision_unix":1779781858.48814,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":70144,"affinity_cache_ratio":0.9949080180985206,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":70144,"estimated_new_tokens":359,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1831,"cache_hit":0,"new_prefill":70503,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":70503,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2495,"cache_hit":70144,"new_prefill":359,"score_linear":-70144.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2241,"cache_hit":0,"new_prefill":70503,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":70503,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":70503,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":69479,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":70503,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-70144.0,"chosen_score_lmetric":0,"t_first_token":350993.651195378,"t_first_token_unix":1779781858.6651387,"t_done":350994.561888792,"t_done_unix":1779781859.5758321},{"request_id":"1340290:48:2033065:1183","session_id":"1340290","input_length":70851,"t_proxy_recv":351008.201122561,"t_decision_unix":1779781873.2150636,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":70144,"affinity_cache_ratio":0.9900213123315126,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":70144,"estimated_new_tokens":707,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1831,"cache_hit":0,"new_prefill":70851,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":70851,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2495,"cache_hit":70144,"new_prefill":707,"score_linear":-70144.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2241,"cache_hit":0,"new_prefill":70851,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":70851,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":70851,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":69827,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":70851,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-70144.0,"chosen_score_lmetric":0,"t_first_token":351008.43138755,"t_first_token_unix":1779781873.4453309,"t_done":351009.081699663,"t_done_unix":1779781874.0956428},{"request_id":"1340290:48:2034908:1184","session_id":"1340290","input_length":71067,"t_proxy_recv":351014.427773028,"t_decision_unix":1779781879.4417145,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":70656,"affinity_cache_ratio":0.9942167250622652,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":70656,"estimated_new_tokens":411,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1831,"cache_hit":0,"new_prefill":71067,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":71067,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2496,"cache_hit":70656,"new_prefill":411,"score_linear":-70656.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2241,"cache_hit":0,"new_prefill":71067,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":71067,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":71067,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":70043,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":71067,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-70656.0,"chosen_score_lmetric":0,"t_first_token":351014.588240103,"t_first_token_unix":1779781879.6021836,"t_done":351015.113489407,"t_done_unix":1779781880.1274328},{"request_id":"1258908:2:2035438:1185","session_id":"1258908","input_length":7844,"t_proxy_recv":351016.232968199,"t_decision_unix":1779781881.2469094,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":7680,"affinity_cache_ratio":0.9790922998470168,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":7680,"estimated_new_tokens":164,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1831,"cache_hit":0,"new_prefill":7844,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":7844,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2496,"cache_hit":0,"new_prefill":7844,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2241,"cache_hit":7680,"new_prefill":164,"score_linear":-7680.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":7844,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":7844,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":7844,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":7844,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-7680.0,"chosen_score_lmetric":0,"t_first_token":351016.264740246,"t_first_token_unix":1779781881.2786837,"t_done":351017.300109817,"t_done_unix":1779781882.314053},{"request_id":"1258908:3:1388683:699","session_id":"1258908","input_length":9120,"t_proxy_recv":351017.303449453,"t_decision_unix":1779781882.3173912,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":7680,"affinity_cache_ratio":0.8421052631578947,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":7680,"estimated_new_tokens":1440,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1831,"cache_hit":0,"new_prefill":9120,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":9120,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2496,"cache_hit":0,"new_prefill":9120,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2241,"cache_hit":7680,"new_prefill":1440,"score_linear":-7680.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":9120,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":9120,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":9120,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":9120,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-7680.0,"chosen_score_lmetric":0,"t_first_token":351017.448054542,"t_first_token_unix":1779781882.4619975,"t_done":351018.131326873,"t_done_unix":1779781883.145271},{"request_id":"1340290:48:2036994:1186","session_id":"1340290","input_length":71240,"t_proxy_recv":351021.365337443,"t_decision_unix":1779781886.3792787,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":70656,"affinity_cache_ratio":0.9918023582257159,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":70656,"estimated_new_tokens":584,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1831,"cache_hit":0,"new_prefill":71240,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":71240,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2496,"cache_hit":70656,"new_prefill":584,"score_linear":-70656.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2244,"cache_hit":0,"new_prefill":71240,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":71240,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":71240,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":70216,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":71240,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-70656.0,"chosen_score_lmetric":0,"t_first_token":351021.545186323,"t_first_token_unix":1779781886.55913,"t_done":351022.276569818,"t_done_unix":1779781887.290513},{"request_id":"1340290:48:2040473:1187","session_id":"1340290","input_length":71545,"t_proxy_recv":351032.891307929,"t_decision_unix":1779781897.9052489,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":71168,"affinity_cache_ratio":0.9947305891397022,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":71168,"estimated_new_tokens":377,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1831,"cache_hit":0,"new_prefill":71545,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":71545,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2497,"cache_hit":71168,"new_prefill":377,"score_linear":-71168.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2244,"cache_hit":0,"new_prefill":71545,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":71545,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":71545,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":70521,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":71545,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-71168.0,"chosen_score_lmetric":0,"t_first_token":351033.123567866,"t_first_token_unix":1779781898.137512,"t_done":351034.284087105,"t_done_unix":1779781899.2980309},{"request_id":"1340290:48:2043771:1188","session_id":"1340290","input_length":71867,"t_proxy_recv":351043.913027761,"t_decision_unix":1779781908.926969,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":71168,"affinity_cache_ratio":0.9902737000292207,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":71168,"estimated_new_tokens":699,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1831,"cache_hit":0,"new_prefill":71867,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":71867,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2497,"cache_hit":71168,"new_prefill":699,"score_linear":-71168.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2244,"cache_hit":0,"new_prefill":71867,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":71867,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":71867,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":70843,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":71867,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-71168.0,"chosen_score_lmetric":0,"t_first_token":351044.143721831,"t_first_token_unix":1779781909.1576655,"t_done":351044.730213554,"t_done_unix":1779781909.744157},{"request_id":"1340290:48:2045403:1189","session_id":"1340290","input_length":72025,"t_proxy_recv":351049.422861187,"t_decision_unix":1779781914.4368024,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":71680,"affinity_cache_ratio":0.995209996528983,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":71680,"estimated_new_tokens":345,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1831,"cache_hit":0,"new_prefill":72025,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":72025,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2498,"cache_hit":71680,"new_prefill":345,"score_linear":-71680.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2244,"cache_hit":0,"new_prefill":72025,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":72025,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":72025,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":71001,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":72025,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-71680.0,"chosen_score_lmetric":0,"t_first_token":351049.562476904,"t_first_token_unix":1779781914.5764203,"t_done":351050.260394794,"t_done_unix":1779781915.2743378},{"request_id":"1340290:48:2047167:1190","session_id":"1340290","input_length":72211,"t_proxy_recv":351055.187663173,"t_decision_unix":1779781920.2016046,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":71680,"affinity_cache_ratio":0.9926465496946448,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":71680,"estimated_new_tokens":531,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1831,"cache_hit":0,"new_prefill":72211,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":72211,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2498,"cache_hit":71680,"new_prefill":531,"score_linear":-71680.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2244,"cache_hit":0,"new_prefill":72211,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":72211,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":72211,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":71187,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":72211,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-71680.0,"chosen_score_lmetric":0,"t_first_token":351055.373343947,"t_first_token_unix":1779781920.3872874,"t_done":351055.934157938,"t_done_unix":1779781920.948101},{"request_id":"1340290:48:2048793:1191","session_id":"1340290","input_length":72777,"t_proxy_recv":351060.624711016,"t_decision_unix":1779781925.6386523,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":72192,"affinity_cache_ratio":0.9919617461560658,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":72192,"estimated_new_tokens":585,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1831,"cache_hit":0,"new_prefill":72777,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":72777,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2499,"cache_hit":72192,"new_prefill":585,"score_linear":-72192.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2244,"cache_hit":0,"new_prefill":72777,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":72777,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":72777,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":71753,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":72777,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-72192.0,"chosen_score_lmetric":0,"t_first_token":351060.997151165,"t_first_token_unix":1779781926.0110943,"t_done":351061.529813688,"t_done_unix":1779781926.543757},{"request_id":"1340290:48:2050838:1192","session_id":"1340290","input_length":72952,"t_proxy_recv":351067.840978353,"t_decision_unix":1779781932.8549194,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":72704,"affinity_cache_ratio":0.9966005044412765,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":72704,"estimated_new_tokens":248,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1831,"cache_hit":0,"new_prefill":72952,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":72952,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2500,"cache_hit":72704,"new_prefill":248,"score_linear":-72704.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2244,"cache_hit":0,"new_prefill":72952,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":72952,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":72952,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":71928,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":72952,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-72704.0,"chosen_score_lmetric":0,"t_first_token":351068.022245436,"t_first_token_unix":1779781933.0361886,"t_done":351068.392660009,"t_done_unix":1779781933.406603},{"request_id":"1340290:48:2052523:1193","session_id":"1340290","input_length":73208,"t_proxy_recv":351073.409671794,"t_decision_unix":1779781938.423613,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":72704,"affinity_cache_ratio":0.9931155065020216,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":72704,"estimated_new_tokens":504,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1831,"cache_hit":0,"new_prefill":73208,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":73208,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2500,"cache_hit":72704,"new_prefill":504,"score_linear":-72704.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2244,"cache_hit":0,"new_prefill":73208,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":73208,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":73208,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":72184,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":73208,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-72704.0,"chosen_score_lmetric":0,"t_first_token":351073.661560932,"t_first_token_unix":1779781938.6755044,"t_done":351074.437188518,"t_done_unix":1779781939.4511316},{"request_id":"1340290:48:2055365:1194","session_id":"1340290","input_length":73454,"t_proxy_recv":351083.123084601,"t_decision_unix":1779781948.1370258,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":73216,"affinity_cache_ratio":0.9967598769297792,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":73216,"estimated_new_tokens":238,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1831,"cache_hit":0,"new_prefill":73454,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":73454,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2501,"cache_hit":73216,"new_prefill":238,"score_linear":-73216.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2244,"cache_hit":0,"new_prefill":73454,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":73454,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":73454,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":72430,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":73454,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-73216.0,"chosen_score_lmetric":0,"t_first_token":351083.371545186,"t_first_token_unix":1779781948.385489,"t_done":351083.784408655,"t_done_unix":1779781948.7983518},{"request_id":"1340290:48:2057042:1195","session_id":"1340290","input_length":73636,"t_proxy_recv":351088.495081061,"t_decision_unix":1779781953.5090222,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":73216,"affinity_cache_ratio":0.9942962681297192,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":73216,"estimated_new_tokens":420,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1831,"cache_hit":0,"new_prefill":73636,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":73636,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2501,"cache_hit":73216,"new_prefill":420,"score_linear":-73216.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2244,"cache_hit":0,"new_prefill":73636,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":73636,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":73636,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":72612,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":73636,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-73216.0,"chosen_score_lmetric":0,"t_first_token":351088.659336487,"t_first_token_unix":1779781953.6732802,"t_done":351088.936829815,"t_done_unix":1779781953.9507732},{"request_id":"1340290:48:2058614:1196","session_id":"1340290","input_length":73653,"t_proxy_recv":351093.639845071,"t_decision_unix":1779781958.6537862,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":73216,"affinity_cache_ratio":0.9940667725686666,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":73216,"estimated_new_tokens":437,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1831,"cache_hit":0,"new_prefill":73653,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":73653,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2501,"cache_hit":73216,"new_prefill":437,"score_linear":-73216.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2244,"cache_hit":0,"new_prefill":73653,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":73653,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":73653,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":72629,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":73653,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-73216.0,"chosen_score_lmetric":0,"t_first_token":351093.734425366,"t_first_token_unix":1779781958.7483687,"t_done":351095.510847383,"t_done_unix":1779781960.5247905},{"request_id":"1340290:48:2061834:1197","session_id":"1340290","input_length":73858,"t_proxy_recv":351103.786435694,"t_decision_unix":1779781968.800377,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":73728,"affinity_cache_ratio":0.9982398656882125,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":73728,"estimated_new_tokens":130,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1831,"cache_hit":0,"new_prefill":73858,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":73858,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2502,"cache_hit":73728,"new_prefill":130,"score_linear":-73728.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2244,"cache_hit":0,"new_prefill":73858,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":73858,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":73858,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":72834,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":73858,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-73728.0,"chosen_score_lmetric":0,"t_first_token":351103.892200427,"t_first_token_unix":1779781968.906144,"t_done":351104.452037859,"t_done_unix":1779781969.4659812},{"request_id":"1258908:3:2063441:1198","session_id":"1258908","input_length":9359,"t_proxy_recv":351108.828557769,"t_decision_unix":1779781973.8424988,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":9216,"affinity_cache_ratio":0.9847205898066033,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":9216,"estimated_new_tokens":143,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1831,"cache_hit":0,"new_prefill":9359,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":9359,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2502,"cache_hit":0,"new_prefill":9359,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2244,"cache_hit":9216,"new_prefill":143,"score_linear":-9216.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":9359,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":9359,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":9359,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":9359,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-9216.0,"chosen_score_lmetric":0,"t_first_token":351108.87899093,"t_first_token_unix":1779781973.892934,"t_done":351109.966585247,"t_done_unix":1779781974.9805286},{"request_id":"1340290:48:2063837:1199","session_id":"1340290","input_length":74011,"t_proxy_recv":351110.179388871,"t_decision_unix":1779781975.1933305,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":73728,"affinity_cache_ratio":0.9961762440718271,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":73728,"estimated_new_tokens":283,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1831,"cache_hit":0,"new_prefill":74011,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":74011,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2502,"cache_hit":73728,"new_prefill":283,"score_linear":-73728.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":9740,"ongoing_decode_tokens":9740,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":2244,"cache_hit":0,"new_prefill":74011,"score_linear":9740.0,"score_lmetric":74011},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":74011,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":74011,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":72987,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":74011,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-73728.0,"chosen_score_lmetric":0,"t_first_token":351110.336319673,"t_first_token_unix":1779781975.3502629,"t_done":351110.90891472,"t_done_unix":1779781975.9228582},{"request_id":"1258908:4:1553508:958","session_id":"1258908","input_length":9740,"t_proxy_recv":351109.970347569,"t_decision_unix":1779781974.9842892,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":9216,"affinity_cache_ratio":0.9462012320328542,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":9216,"estimated_new_tokens":524,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1831,"cache_hit":0,"new_prefill":9740,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":9740,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2502,"cache_hit":0,"new_prefill":9740,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2244,"cache_hit":9216,"new_prefill":524,"score_linear":-9216.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":9740,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":9740,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":9740,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":9740,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-9216.0,"chosen_score_lmetric":0,"t_first_token":351110.027242439,"t_first_token_unix":1779781975.0411866,"t_done":351111.949812158,"t_done_unix":1779781976.9637568},{"request_id":"1258908:5:1617181:988","session_id":"1258908","input_length":10201,"t_proxy_recv":351111.953186927,"t_decision_unix":1779781976.9671285,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":3,"chosen_idx":3,"affinity_cache_hit":9728,"affinity_cache_ratio":0.9536319968630527,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":9728,"estimated_new_tokens":473,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8003","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1831,"cache_hit":0,"new_prefill":10201,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":10201,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2502,"cache_hit":0,"new_prefill":10201,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2245,"cache_hit":9728,"new_prefill":473,"score_linear":-9728.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":10201,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":10201,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":10201,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":10201,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-9728.0,"chosen_score_lmetric":0,"t_first_token":351112.001830541,"t_first_token_unix":1779781977.0157738,"t_done":351114.091229607,"t_done_unix":1779781979.1051726},{"request_id":"1340290:48:2066647:1200","session_id":"1340290","input_length":74557,"t_proxy_recv":351119.61748219,"t_decision_unix":1779781984.6314235,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":73728,"affinity_cache_ratio":0.9888809903831968,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":73728,"estimated_new_tokens":829,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1831,"cache_hit":0,"new_prefill":74557,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":74557,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2502,"cache_hit":73728,"new_prefill":829,"score_linear":-73728.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2246,"cache_hit":0,"new_prefill":74557,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":74557,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":74557,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":73533,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":74557,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-73728.0,"chosen_score_lmetric":0,"t_first_token":351120.022933896,"t_first_token_unix":1779781985.0368774,"t_done":351121.260220676,"t_done_unix":1779781986.2741642},{"request_id":"1340290:48:2071598:1201","session_id":"1340290","input_length":74863,"t_proxy_recv":351136.262847516,"t_decision_unix":1779782001.2767882,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":74240,"affinity_cache_ratio":0.991678132054553,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":74240,"estimated_new_tokens":623,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1831,"cache_hit":0,"new_prefill":74863,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":74863,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2503,"cache_hit":74240,"new_prefill":623,"score_linear":-74240.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2246,"cache_hit":0,"new_prefill":74863,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":74863,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":74863,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":73839,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":74863,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-74240.0,"chosen_score_lmetric":0,"t_first_token":351136.505383979,"t_first_token_unix":1779782001.5193276,"t_done":351137.326526929,"t_done_unix":1779782002.3404706},{"request_id":"1340290:48:2076822:1202","session_id":"1340290","input_length":75113,"t_proxy_recv":351153.795166968,"t_decision_unix":1779782018.8091078,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":74752,"affinity_cache_ratio":0.9951939078455128,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":74752,"estimated_new_tokens":361,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1831,"cache_hit":0,"new_prefill":75113,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":75113,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2504,"cache_hit":74752,"new_prefill":361,"score_linear":-74752.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2246,"cache_hit":0,"new_prefill":75113,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":75113,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":75113,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":74089,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":75113,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-74752.0,"chosen_score_lmetric":0,"t_first_token":351154.036346639,"t_first_token_unix":1779782019.05029,"t_done":351154.481801494,"t_done_unix":1779782019.4957447},{"request_id":"1340290:48:2078461:1203","session_id":"1340290","input_length":75299,"t_proxy_recv":351159.080548758,"t_decision_unix":1779782024.0944898,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":74752,"affinity_cache_ratio":0.9927356272991673,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":74752,"estimated_new_tokens":547,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1831,"cache_hit":0,"new_prefill":75299,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":75299,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2504,"cache_hit":74752,"new_prefill":547,"score_linear":-74752.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2246,"cache_hit":0,"new_prefill":75299,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":75299,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":75299,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":74275,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":75299,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-74752.0,"chosen_score_lmetric":0,"t_first_token":351159.312743621,"t_first_token_unix":1779782024.3266866,"t_done":351159.604568724,"t_done_unix":1779782024.618512},{"request_id":"1340290:48:2080016:1204","session_id":"1340290","input_length":75338,"t_proxy_recv":351164.370297324,"t_decision_unix":1779782029.3842385,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":75264,"affinity_cache_ratio":0.9990177599617723,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":75264,"estimated_new_tokens":74,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1831,"cache_hit":0,"new_prefill":75338,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":75338,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2505,"cache_hit":75264,"new_prefill":74,"score_linear":-75264.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2246,"cache_hit":0,"new_prefill":75338,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":75338,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":75338,"score_linear":0.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":74314,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":75338,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-75264.0,"chosen_score_lmetric":0,"t_first_token":351164.476479894,"t_first_token_unix":1779782029.4904234,"t_done":351164.852771109,"t_done_unix":1779782029.8667145},{"request_id":"1340290:48:2082644:1206","session_id":"1340290","input_length":75439,"t_proxy_recv":351173.048445954,"t_decision_unix":1779782038.0623875,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":75264,"affinity_cache_ratio":0.9976802449661316,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":75264,"estimated_new_tokens":175,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1831,"cache_hit":0,"new_prefill":75439,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":75439,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2505,"cache_hit":75264,"new_prefill":175,"score_linear":-75264.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2246,"cache_hit":0,"new_prefill":75439,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":75439,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":114971,"ongoing_decode_tokens":0,"pending_prefill_tokens":61211,"num_requests":1,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":75439,"score_linear":114971.0,"score_lmetric":136650},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":74415,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":75439,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-75264.0,"chosen_score_lmetric":0,"t_first_token":351173.199996903,"t_first_token_unix":1779782038.2139401,"t_done":351174.200382916,"t_done_unix":1779782039.2143264},{"request_id":"1356885:16:2084239:1207","session_id":"1356885","input_length":136991,"t_proxy_recv":351178.141439874,"t_decision_unix":1779782043.1553807,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":136704,"affinity_cache_ratio":0.9979049718594653,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":136704,"estimated_new_tokens":287,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1831,"cache_hit":136704,"new_prefill":287,"score_linear":-136704.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":1024,"new_prefill":135967,"score_linear":-1024.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2505,"cache_hit":0,"new_prefill":136991,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2246,"cache_hit":0,"new_prefill":136991,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":6656,"new_prefill":130335,"score_linear":-6656.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":114971,"ongoing_decode_tokens":0,"pending_prefill_tokens":61211,"num_requests":1,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":136991,"score_linear":114971.0,"score_lmetric":198202},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":136991,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":136991,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-136704.0,"chosen_score_lmetric":0,"t_first_token":351178.557512123,"t_first_token_unix":1779782043.5714562,"t_done":351181.40190896,"t_done_unix":1779782046.4158525},{"request_id":"1340290:48:2085626:1208","session_id":"1340290","input_length":77337,"t_proxy_recv":351183.126879357,"t_decision_unix":1779782048.1408205,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":75264,"affinity_cache_ratio":0.9731952364327553,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":75264,"estimated_new_tokens":2073,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1832,"cache_hit":0,"new_prefill":77337,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":77337,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2505,"cache_hit":75264,"new_prefill":2073,"score_linear":-75264.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2246,"cache_hit":0,"new_prefill":77337,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":77337,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":114971,"ongoing_decode_tokens":0,"pending_prefill_tokens":61211,"num_requests":1,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":77337,"score_linear":114971.0,"score_lmetric":138548},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":76313,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":77337,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-75264.0,"chosen_score_lmetric":0,"t_first_token":351184.101790448,"t_first_token_unix":1779782049.1157343,"t_done":351184.618325112,"t_done_unix":1779782049.6322684},{"request_id":"1340290:48:2087584:1209","session_id":"1340290","input_length":78683,"t_proxy_recv":351189.699556505,"t_decision_unix":1779782054.7134976,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":77312,"affinity_cache_ratio":0.9825756516655441,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":77312,"estimated_new_tokens":1371,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1832,"cache_hit":0,"new_prefill":78683,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":78683,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2509,"cache_hit":77312,"new_prefill":1371,"score_linear":-77312.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2246,"cache_hit":0,"new_prefill":78683,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":78683,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":114971,"ongoing_decode_tokens":0,"pending_prefill_tokens":61211,"num_requests":1,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":78683,"score_linear":114971.0,"score_lmetric":139894},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":77659,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":78683,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-77312.0,"chosen_score_lmetric":0,"t_first_token":351190.531648109,"t_first_token_unix":1779782055.545592,"t_done":351192.015801204,"t_done_unix":1779782057.0297446},{"request_id":"1340290:48:2092759:1211","session_id":"1340290","input_length":79023,"t_proxy_recv":351206.983612897,"t_decision_unix":1779782071.9975538,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":2,"chosen_idx":2,"affinity_cache_hit":78336,"affinity_cache_ratio":0.9913063285372613,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":78336,"estimated_new_tokens":687,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8002","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1832,"cache_hit":0,"new_prefill":79023,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":79023,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2511,"cache_hit":78336,"new_prefill":687,"score_linear":-78336.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2246,"cache_hit":0,"new_prefill":79023,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":79023,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":114971,"ongoing_decode_tokens":0,"pending_prefill_tokens":61211,"num_requests":1,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":0,"new_prefill":79023,"score_linear":114971.0,"score_lmetric":140234},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":1024,"new_prefill":77999,"score_linear":-1024.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":79023,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-78336.0,"chosen_score_lmetric":0,"t_first_token":351207.234012462,"t_first_token_unix":1779782072.2479572,"t_done":351208.536096993,"t_done_unix":1779782073.5500414},{"request_id":"1382086:2:2082052:1205","session_id":"1382086","input_length":114971,"t_proxy_recv":351171.026564831,"t_decision_unix":1779782036.0405056,"policy":"unified_nixl_both","decision":"lmetric_fallback","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":53760,"affinity_cache_ratio":0.467596176427099,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":0,"tie_break_used":false,"cache_hit":53760,"estimated_new_tokens":61211,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1831,"cache_hit":0,"new_prefill":114971,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":114971,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2505,"cache_hit":0,"new_prefill":114971,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2246,"cache_hit":0,"new_prefill":114971,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":114971,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1800,"cache_hit":53760,"new_prefill":61211,"score_linear":-53760.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":114971,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":114971,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-53760.0,"chosen_score_lmetric":0,"t_first_token":351213.45608736,"t_first_token_unix":1779782078.4700313,"t_done":351214.63922548,"t_done_unix":1779782079.653169},{"request_id":"1382086:3:2089879:1210","session_id":"1382086","input_length":115184,"t_proxy_recv":351214.657845595,"t_decision_unix":1779782079.6717873,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":114688,"affinity_cache_ratio":0.9956938463675511,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":114688,"estimated_new_tokens":496,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1832,"cache_hit":0,"new_prefill":115184,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":115184,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2512,"cache_hit":0,"new_prefill":115184,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2246,"cache_hit":0,"new_prefill":115184,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":115184,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1919,"cache_hit":114688,"new_prefill":496,"score_linear":-114688.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":115184,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":115184,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-114688.0,"chosen_score_lmetric":0,"t_first_token":351214.932570134,"t_first_token_unix":1779782079.946514,"t_done":351215.451182753,"t_done_unix":1779782080.4651265},{"request_id":"1382086:4:2092953:1212","session_id":"1382086","input_length":115246,"t_proxy_recv":351215.470466281,"t_decision_unix":1779782080.484408,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":5,"chosen_idx":5,"affinity_cache_hit":115200,"affinity_cache_ratio":0.9996008538257293,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":115200,"estimated_new_tokens":46,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8005","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1832,"cache_hit":0,"new_prefill":115246,"score_linear":0.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":0,"new_prefill":115246,"score_linear":0.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2512,"cache_hit":0,"new_prefill":115246,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2246,"cache_hit":0,"new_prefill":115246,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":0,"new_prefill":115246,"score_linear":0.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1920,"cache_hit":115200,"new_prefill":46,"score_linear":-115200.0,"score_lmetric":0},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":115246,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":115246,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-115200.0,"chosen_score_lmetric":0,"t_first_token":351215.615809274,"t_first_token_unix":1779782080.6297526,"t_done":351222.727030443,"t_done_unix":1779782087.7409751},{"request_id":"1356885:17:2097011:1213","session_id":"1356885","input_length":137546,"t_proxy_recv":351220.857035283,"t_decision_unix":1779782085.8709764,"policy":"unified_nixl_both","decision":"affinity","affinity_idx":0,"chosen_idx":0,"affinity_cache_hit":137216,"affinity_cache_ratio":0.9976008026405712,"affinity_num_requests":0,"avg_num_requests":1.0,"fallback_score":null,"tie_break_used":false,"cache_hit":137216,"estimated_new_tokens":330,"route_class":"LOCAL","routed_to":"http://127.0.0.1:8000","candidate_scores":[{"idx":0,"url":"http://127.0.0.1:8000","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1832,"cache_hit":137216,"new_prefill":330,"score_linear":-137216.0,"score_lmetric":0},{"idx":1,"url":"http://127.0.0.1:8001","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2041,"cache_hit":1024,"new_prefill":136522,"score_linear":-1024.0,"score_lmetric":0},{"idx":2,"url":"http://127.0.0.1:8002","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2512,"cache_hit":0,"new_prefill":137546,"score_linear":0.0,"score_lmetric":0},{"idx":3,"url":"http://127.0.0.1:8003","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2246,"cache_hit":0,"new_prefill":137546,"score_linear":0.0,"score_lmetric":0},{"idx":4,"url":"http://127.0.0.1:8004","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":2767,"cache_hit":6656,"new_prefill":130890,"score_linear":-6656.0,"score_lmetric":0},{"idx":5,"url":"http://127.0.0.1:8005","ongoing_tokens":115246,"ongoing_decode_tokens":115246,"pending_prefill_tokens":0,"num_requests":1,"active_p_offloads":0,"cached_blocks":1920,"cache_hit":0,"new_prefill":137546,"score_linear":115246.0,"score_lmetric":137546},{"idx":6,"url":"http://127.0.0.1:8006","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1711,"cache_hit":0,"new_prefill":137546,"score_linear":0.0,"score_lmetric":0},{"idx":7,"url":"http://127.0.0.1:8007","ongoing_tokens":0,"ongoing_decode_tokens":0,"pending_prefill_tokens":0,"num_requests":0,"active_p_offloads":0,"cached_blocks":1730,"cache_hit":0,"new_prefill":137546,"score_linear":0.0,"score_lmetric":0}],"chosen_score_linear":-137216.0,"chosen_score_lmetric":0,"t_first_token":351221.513588544,"t_first_token_unix":1779782086.5275326,"t_done":351224.208765399,"t_done_unix":1779782089.2227094}]
\ No newline at end of file
diff --git a/analysis/characterization/elastic_migration_v2/data/per_worker_unified_nixl_both.json b/analysis/characterization/elastic_migration_v2/data/per_worker_unified_nixl_both.json
new file mode 100644
index 0000000..8f38c1b
--- /dev/null
+++ b/analysis/characterization/elastic_migration_v2/data/per_worker_unified_nixl_both.json
@@ -0,0 +1,24 @@
+{
+  "hotspot_index_ttft_p90": 3.673957447190547,
+  "per_worker_latency_p90_s": {
+    "http://127.0.0.1:8000": 21.5702620673168,
+    "http://127.0.0.1:8001": 21.44246501957532,
+    "http://127.0.0.1:8002": 7.497513776784784,
+    "http://127.0.0.1:8003": 18.975387462502113,
+    "http://127.0.0.1:8004": 27.733961877820548,
+    "http://127.0.0.1:8005": 14.178356938017535,
+    "http://127.0.0.1:8006": 25.44877168269595,
+    "http://127.0.0.1:8007": 54.500166546402035
+  },
+  "per_worker_ttft_p90_s": {
+    "http://127.0.0.1:8000": 7.380765471985799,
+    "http://127.0.0.1:8001": 14.109222683508415,
+    "http://127.0.0.1:8002": 3.001173847797329,
+    "http://127.0.0.1:8003": 14.087287129514152,
+    "http://127.0.0.1:8004": 14.151121024426537,
+    "http://127.0.0.1:8005": 6.165523712011057,
+    "http://127.0.0.1:8006": 6.314287615299688,
+    "http://127.0.0.1:8007": 39.43635586597957
+  },
+  "status": "supported"
+}
\ No newline at end of file
diff --git a/analysis/characterization/elastic_migration_v2/figures/fig_connector_substrate_attribution.png b/analysis/characterization/elastic_migration_v2/figures/fig_connector_substrate_attribution.png
new file mode 100644
index 0000000..42d7558
Binary files /dev/null and b/analysis/characterization/elastic_migration_v2/figures/fig_connector_substrate_attribution.png differ
diff --git a/analysis/characterization/elastic_migration_v2/figures/fig_kv_both_overhead.png b/analysis/characterization/elastic_migration_v2/figures/fig_kv_both_overhead.png
index 7f363ef..53e29e1 100644
Binary files a/analysis/characterization/elastic_migration_v2/figures/fig_kv_both_overhead.png and b/analysis/characterization/elastic_migration_v2/figures/fig_kv_both_overhead.png differ
diff --git a/analysis/characterization/elastic_migration_v2/figures/fig_three_way_hotspot.png b/analysis/characterization/elastic_migration_v2/figures/fig_three_way_hotspot.png
index dcad0a0..feeba9d 100644
Binary files a/analysis/characterization/elastic_migration_v2/figures/fig_three_way_hotspot.png and b/analysis/characterization/elastic_migration_v2/figures/fig_three_way_hotspot.png differ
diff --git a/analysis/characterization/elastic_migration_v2/render_figures.py b/analysis/characterization/elastic_migration_v2/render_figures.py
index 123f261..3fd8742 100644
--- a/analysis/characterization/elastic_migration_v2/render_figures.py
+++ b/analysis/characterization/elastic_migration_v2/render_figures.py
@@ -34,37 +34,39 @@ def _load(name: str):
 
 
 POLICY_COLORS = {
-    "unified":            "#2ca02c",
-    "unified_kv_both":    "#9467bd",
-    "unified_v2":         "#d62728",
-    "unified_v2_strict":  "#ff7f0e",
+    "unified":             "#2ca02c",
+    "unified_kv_both":     "#9467bd",
+    "unified_nixl_both":   "#1f77b4",
+    "unified_v2":          "#d62728",
+    "unified_v2_strict":   "#ff7f0e",
 }
 
 
 def fig_kv_both_overhead():
     comp = _load("b3_policy_comparison.json")
     by = {r["policy"]: r for r in comp["rows"]}
-    pols = ["unified", "unified_kv_both", "unified_v2"]
+    pols = ["unified", "unified_kv_both", "unified_nixl_both", "unified_v2"]
     metrics = [
         ("TTFT p90 (s)",   lambda r: r["ttft_p90_s"]),
         ("TPOT p90 (ms)",  lambda r: r["tpot_p90_s"] * 1000),
         ("E2E p90 (s)",    lambda r: r["e2e_p90_s"]),
         ("hotspot index",  lambda r: r["hotspot_index_ttft_p90"]),
     ]
-    fig, axes = plt.subplots(1, 4, figsize=(14, 4))
+    fig, axes = plt.subplots(1, 4, figsize=(15, 4.2))
     for ax, (label, fn) in zip(axes, metrics):
         vals = [fn(by[p]) for p in pols]
-        bars = ax.bar(pols, vals,
+        labels_short = [p.replace("unified_", "") for p in pols]
+        labels_short[0] = "plain"
+        bars = ax.bar(labels_short, vals,
                        color=[POLICY_COLORS[p] for p in pols],
                        edgecolor="black", linewidth=0.5)
         ax.set_title(label)
-        ax.tick_params(axis="x", rotation=20, labelsize=9)
+        ax.tick_params(axis="x", rotation=15, labelsize=9)
         for b, v in zip(bars, vals):
             ax.text(b.get_x() + b.get_width() / 2, v,
                      f"{v:.2f}" if v < 100 else f"{v:.0f}",
                      ha="center", va="bottom", fontsize=9)
         ax.grid(alpha=0.3, axis="y")
-        # delta annotation
         baseline = vals[0]
         for i, v in enumerate(vals):
             if i == 0:
@@ -74,8 +76,8 @@ def fig_kv_both_overhead():
                      fontsize=10, fontweight="bold",
                      color="darkred" if pct > 0 else "darkgreen")
     fig.suptitle(
-        "kv_both adds ~45% to TTFT p90 even without PD-sep firing.\n"
-        "v2's PD-sep barely recovers the gap (and overshoots TTFT p99)."
+        "Mooncake substrate adds 19-45% across metrics; NIXL is 5-19pp better but\n"
+        "still 16-38% above plain. v2's 5 PD-sep events don't recover the substrate tax."
     )
     fig.tight_layout()
     fig.savefig(OUT / "fig_kv_both_overhead.png", dpi=120)
@@ -203,27 +205,29 @@ def fig_v2_predicted_vs_actual():
 
 
 def fig_three_way_hotspot():
-    pols = ["unified", "unified_kv_both", "unified_v2"]
+    pols = ["unified", "unified_kv_both", "unified_nixl_both", "unified_v2"]
     per_worker = {p: _load(f"per_worker_{p}.json") for p in pols}
     workers = sorted(per_worker["unified"]["per_worker_ttft_p90_s"].keys())
 
     x = range(len(workers))
-    width = 0.27
-    fig, ax = plt.subplots(figsize=(11, 5))
+    n = len(pols)
+    width = 0.85 / n
+    fig, ax = plt.subplots(figsize=(12, 5))
     for i, p in enumerate(pols):
         d = per_worker[p]["per_worker_ttft_p90_s"]
         vals = [d[w] for w in workers]
-        offset = (i - 1) * width
+        offset = (i - (n - 1) / 2) * width
+        label = p.replace("unified_", "") if p != "unified" else "plain"
         ax.bar([j + offset for j in x], vals, width,
-                label=f"{p} (hotspot={per_worker[p]['hotspot_index_ttft_p90']:.2f})",
+                label=f"{label} (hotspot={per_worker[p]['hotspot_index_ttft_p90']:.2f})",
                 color=POLICY_COLORS[p], edgecolor="black", linewidth=0.4)
     short = [w.replace("http://127.0.0.1:", ":") for w in workers]
     ax.set_xticks(list(x))
     ax.set_xticklabels(short, rotation=0, fontsize=9)
     ax.set_ylabel("worker TTFT p90 (s)")
     ax.set_title(
-        "Per-worker TTFT p90 distribution. kv_both alone makes the hot worker hotter\n"
-        "(unified→kv_both: 37.7s→43.5s peak); v2's 5 PD-sep triggers nudge it back."
+        "Per-worker TTFT p90 distribution across substrates. Mooncake (kv_both)\n"
+        "amplifies the hot worker (hotspot 4.36); NIXL keeps it close to plain (3.67)."
     )
     ax.legend(loc="upper left", fontsize=9)
     ax.grid(alpha=0.3, axis="y")
@@ -232,12 +236,64 @@ def fig_three_way_hotspot():
     plt.close(fig)
 
 
+def fig_connector_substrate_attribution():
+    """Decomposes overhead into v1-framework cost (shared by all connectors,
+    proxied by NIXL since it's the leanest) and Mooncake-specific cost."""
+    comp = _load("b3_policy_comparison.json")
+    by = {r["policy"]: r for r in comp["rows"]}
+    metrics = [
+        ("TTFT p90 (s)",  "ttft_p90_s",  False),
+        ("TPOT p90 (ms)", "tpot_p90_s",  True),
+        ("E2E p90 (s)",   "e2e_p90_s",   False),
+        ("hotspot index", "hotspot_index_ttft_p90", False),
+    ]
+    fig, axes = plt.subplots(1, 4, figsize=(15, 4))
+    for ax, (label, key, scale_ms) in zip(axes, metrics):
+        plain = by["unified"][key] * (1000 if scale_ms else 1)
+        nixl = by["unified_nixl_both"][key] * (1000 if scale_ms else 1)
+        moon = by["unified_kv_both"][key] * (1000 if scale_ms else 1)
+        v2 = by["unified_v2"][key] * (1000 if scale_ms else 1)
+
+        framework_cost = nixl - plain   # what NIXL adds = v1 framework cost
+        mooncake_extra = moon - nixl    # extra on top from Mooncake
+        v2_branch_extra = v2 - moon     # extra from PD-sep branch (Mooncake + 5 events)
+
+        bottom = 0
+        ax.bar(["overhead"], [plain], color="#cccccc",
+                edgecolor="black", linewidth=0.4,
+                label=f"plain unified ({plain:.2f})")
+        bottom += plain
+        ax.bar(["overhead"], [framework_cost], bottom=[bottom],
+                color="#1f77b4", edgecolor="black", linewidth=0.4,
+                label=f"v1 framework (+{framework_cost:.2f})")
+        bottom += framework_cost
+        ax.bar(["overhead"], [mooncake_extra], bottom=[bottom],
+                color="#9467bd", edgecolor="black", linewidth=0.4,
+                label=f"Mooncake extra (+{mooncake_extra:.2f})")
+        bottom += mooncake_extra
+        ax.bar(["overhead"], [v2_branch_extra], bottom=[bottom],
+                color="#d62728", edgecolor="black", linewidth=0.4,
+                label=f"v2 PD-sep branch ({v2_branch_extra:+.2f})")
+        ax.set_title(label)
+        ax.legend(fontsize=8, loc="upper right")
+        ax.grid(alpha=0.3, axis="y")
+        ax.tick_params(axis="x", labelbottom=False)
+    fig.suptitle(
+        "Attribution: plain unified vs NIXL substrate vs Mooncake substrate vs v2.\n"
+        "Blue: cost shared by any v1 connector. Purple: cost specific to Mooncake."
+    )
+    fig.tight_layout()
+    fig.savefig(OUT / "fig_connector_substrate_attribution.png", dpi=120)
+    plt.close(fig)
+
+
 def main():
     fig_kv_both_overhead()
     fig_v2_trigger_funnel()
     fig_v2_predicted_vs_actual()
     fig_three_way_hotspot()
-    print(f"wrote 4 figures to {OUT}")
+    fig_connector_substrate_attribution()
+    print(f"wrote 5 figures to {OUT}")
 
 
 if __name__ == "__main__":