Run, tune, and operate self-hosted LLMs
Blueprint is the open-source toolkit and browser planner we wrote to make private AI infrastructure actually shippable — without paying per-token API bills or letting data leave your network.
Free under Apache 2.0. We offer paid engagements when teams want help applying it to their workloads.
What's in the box
The pieces you actually need to run AI in-house
Multi-engine serving
llama.cpp, vLLM, and TensorRT-LLM behind one OpenAI-compatible endpoint. Pick the engine that fits the hardware; swap without rewriting clients.
Custom quantization
Calibrate quantization against your own workload, not a generic dataset. Recover accuracy lost to off-the-shelf INT4/INT8.
LoRA / QLoRA fine-tuning
Train task-specific adapters in hours instead of full fine-tunes. Stack and hot-swap adapters at serve time.
Semantic routing
Route easy queries to small models and hard ones to large models. Cut spend without losing accuracy on the cases that matter.
Prompt caching & compression
Cache hot prefixes and compress long context with LLMLingua. 40–75% input-token reduction on real workloads.
Browser planner
Size a model, the hardware to run it, and the real spend — on-prem or in the cloud — without installing anything.
Who it's for
Teams that have outgrown the API
- Platform teams building an internal LLM stack their org depends on
- Consultants and practitioners running optimization engagements
- Engineers learning the full pipeline — sizing, serving, fine-tuning, ops
Want help applying Blueprint to your workload?
The toolkit is free. The engagement that takes it from working in dev to running your business is what we sell.