InferenceX

Supercharge Your LLM with the Fastest KV Cache Layer

A high-throughput and memory-efficient inference and serving engine for LLMs

RamaLama is an open-source developer tool that simplifies the local serving of AI models from any source and facilitates their use for inference in production, all through the familiar language of containers.

Machine Learning Engineering Open Book