Back to List
Notice:This resource is provided by a third-party author. Please review the code with AI tools or manually before use to ensure security and compatibility.
C++vllm-project/vllm-ascend

vllm-ascend

Community maintained hardware plugin for vLLM on Ascend

78.2/100
2.0KForks: 1.1K
View on GitHubHomepage →
Loading report...

Similar Projects

Mooncake

84

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

C++5.2K

ZhiLight

65

A highly optimized LLM inference acceleration engine for Llama and its variants.

C++904

vllm

93

A high-throughput and memory-efficient inference and serving engine for LLMs

Python77.8K

whisper.cpp

85

Port of OpenAI's Whisper model in C/C++

C++48.9K
Back to List