← Back to List
⚠
Notice:This resource is provided by a third-party author. Please review the code with AI tools or manually before use to ensure security and compatibility.
C++zhihu/ZhiLight

ZhiLight

A highly optimized LLM inference acceleration engine for Llama and its variants.

54.9/100
★ 905Forks: 103
View on GitHub →
Loading report...

Similar Projects

vllm

93

A high-throughput and memory-efficient inference and serving engine for LLMs

Python★ 87.1K

lorax

73

Multi-LoRA inference server that scales to 1000s of fine-tuned LLMs

Python★ 3.8K

PowerInfer

58

High-speed Large Language Model Serving for Local Deployment

C++★ 9.7K

lemonade

85

Lemonade helps users discover and run local AI apps by serving optimized LLMs right from their own GPUs and NPUs. Join our discord: https://discord.gg/5xXzkMu8Zk

C++★ 5.1K
← Back to List