← Back to List
⚠
Notice:This resource is provided by a third-party author. Please review the code with AI tools or manually before use to ensure security and compatibility.
C++Luce-Org/lucebox

lucebox

LLM speculative inference server for heterogeneous hardware & consumer GPUs

73.5/100
★ 2.9KForks: 282
View on GitHub →Homepage →
Loading report...

Similar Projects

FlashRT

69

FlashRT is a high-performance realtime inference engine for small-batch, latency-sensitive AI workloads. The flagship integration is production VLA control for Pi0, Pi0.5, GROOT N1.6, and Pi0-FAST. Also support llm e.g, qwen3.6-27B

C++★ 615

llamafile

88

Distribute and run LLMs with a single file.

C++★ 26.2K

surogate

82

Train and serve LLMs at extreme speed and massive throughput.

C++★ 849

lemonade

85

Lemonade helps users discover and run local AI apps by serving optimized LLMs right from their own GPUs and NPUs. Join our discord: https://discord.gg/5xXzkMu8Zk

C++★ 5.8K
← Back to List