cactus

Production ready toolkit to run AI locally

Distributed LLM inference. Connect home devices into a powerful cluster to accelerate LLM inference. More devices means faster inference.

Build your own high performance LLM inference engine in C++ and CUDA - a smaller version of vLLM

Local LLM, image&video&music generator, vibecode like cursor with local models on your phone