vllm-project/vllmA high-throughput and memory-efficient inference and serving engine for LLMs
A high-throughput and memory-efficient inference and serving engine for LLMs
93k stars23kforks2.5kissuespushed
licence Apache-2.0since 2023branch mainvllm.ai ↗github.com/vllm-project/vllm ↗
verdict · rules report
Worth your time.
Outside contributors get merged here, though some wait a while for a reply.
16 of 67outside PRs merged (24%)
checked