476 B
476 B
Objectives
- Serverless KVCache cache
- PhOS profile
Key Results
- Implement a workload aware KVCache scheduler. [3/10]
- Provide test apps for PhOS
Last Week
- Implement a simulator for KVCache scheduler to quick test different policies.
- Prepare and do a paper sharing in Ali.
- Provide StableDiffusion single GPU train, Llama2-13b multi GPU train, Llama2-70b multi GPU inference script for PhOS profiling.
Next Week
- Implement a solution to reduce KVCache memory need.