302 B
302 B
Objective
- Serverless KVCache cache
Key Result
- Rebuttal for ATC'25
- Refine cache policy implementation
Last Week
- Finish rebuttal for ATC'25 w/ Jinbo.
- Fix some bugs in our cache policy and test in simulator to get a bit hit ratio improvement.
Next Week
- Implement WA policy in vllm and test.