562 B
562 B
ServeGen: Workload Characterization and Generation of Large Language Model Serving in Production
优先 evict M queue
!
| S3FIFO | |
|---|---|
| 1kGPU1kCPU | 0.095005 |
| 1kGPU2kCPU | 0.136413 |
| 1kGPU4kCPU | 0.213832 |
优先 evict S queue
!
| S3FIFO | |
|---|---|
| 1kGPU1kCPU | 0.095005 |
| 1kGPU2kCPU | 0.136413 |
| 1kGPU4kCPU | 0.213832 |