llm-d/ llm-d
Achieve state of the art inference performance with modern accelerators on Kubernetes
Python · Shell 4.7k stars
3 starter issues1 unclaimed
- code
- docs
replies in about a day
25 first-timers merged
An issue to start with#1659 [Docs][RL] Guide: RL training loop with llm-d serving (no Kubernetes) (opens GitHub)