Back to List
Notice:This resource is provided by a third-party author. Please review the code with AI tools or manually before use to ensure security and compatibility.
C++vllm-project/vllm-ascend

vllm-ascend

Community maintained hardware plugin for vLLM on Ascend

79.1/100
2.8KForks: 2.2K
View on GitHubHomepage →
Loading report...

Similar Projects

Mooncake

88

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

C++6.5K

ZhiLight

53

A highly optimized LLM inference acceleration engine for Llama and its variants.

C++908

vllm

93

A high-throughput and memory-efficient inference and serving engine for LLMs

Python91.2K

whisper.cpp

87

Port of OpenAI's Whisper model in C/C++

C++53.5K
Back to List