15 lines
476 B
Markdown
15 lines
476 B
Markdown
Objectives
|
|
- Serverless KVCache cache
|
|
- PhOS profile
|
|
|
|
Key Results
|
|
- Implement a workload aware KVCache scheduler. [3/10]
|
|
- Provide test apps for PhOS
|
|
|
|
Last Week
|
|
- Implement a simulator for KVCache scheduler to quick test different policies.
|
|
- Prepare and do a paper sharing in Ali.
|
|
- Provide StableDiffusion single GPU train, Llama2-13b multi GPU train, Llama2-70b multi GPU inference script for PhOS profiling.
|
|
|
|
Next Week
|
|
- Implement a solution to reduce KVCache memory need. |