Back to List
Notice:This resource is provided by a third-party author. Please review the code with AI tools or manually before use to ensure security and compatibility.
C++vllm-project/vllm-ascend

vllm-ascend

Community maintained hardware plugin for vLLM on Ascend

78.8/100
2.5KForks: 1.8K
View on GitHubHomepage →
Loading report...

Similar Projects

Mooncake

87

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

C++6.0K

ZhiLight

55

A highly optimized LLM inference acceleration engine for Llama and its variants.

C++905

vllm

93

A high-throughput and memory-efficient inference and serving engine for LLMs

Python87.1K

whisper.cpp

81

Port of OpenAI's Whisper model in C/C++

C++52.3K
Back to List