sglang

A high-throughput and memory-efficient inference and serving engine for LLMs

A GPU cluster manager that configures and orchestrates inference engines like vLLM and SGLang for high-performance AI model deployment.

TokenSpeed is a speed-of-light LLM inference engine.

🚀 Pytorch Distributed native training library for LLMs/VLMs with OOTB Hugging Face support