vllm-ascend

Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.

A highly optimized LLM inference acceleration engine for Llama and its variants.

A high-performance inference system for large language models, designed for production environments.

A high-throughput and memory-efficient inference and serving engine for LLMs