vLLM
vLLM is a high-throughput, memory-efficient inference and serving engine for large language models.
Website: vllm.ai
Capabilities
- vLLM (Website): vLLM is a high-throughput, memory-efficient inference and serving engine for large language models.