← Back to List
⚠
Notice:This resource is provided by a third-party author. Please review the code with AI tools or manually before use to ensure security and compatibility.
C++Luce-Org/lucebox

lucebox

LLM speculative inference server for heterogeneous hardware & consumer GPUs

72.7/100
★ 2.8KForks: 273
View on GitHub →Homepage →
Loading report...

Similar Projects

FlashRT

67

FlashRT is a high-performance realtime inference engine for small-batch, latency-sensitive AI workloads. The flagship integration is production VLA control for Pi0, Pi0.5, GROOT N1.6, and Pi0-FAST. Also support llm e.g, qwen3.6-27B

C++★ 553

llamafile

87

Distribute and run LLMs with a single file.

C++★ 25.9K

RCLI

84

Talk to your Mac, query your docs, no cloud required. On-device voice AI + RAG

C++★ 1.5K

surogate

78

Training/Fine-tuning at the speed of light

C++★ 813
← Back to List