Zero-latency proxy for your LLM stack.

Drop-in Rust proxy. Exact + semantic caching in pure Rust.
Cut redundant API calls to zero. One line of config to test.
✗ raw openai — 100 identical prompts avg 1.2s per call · $0.42 total
✓ via stackintercept (cache hit) avg 1.8ms per call · $0.00 API cost
→ 99.8% latency eliminated
<2ms
exact cache overhead
133 MB
local BGE model, no API
$0
cost on cache hits
$
Drop-in. Change base_url in your OpenAI/Anthropic SDK client. That's it.
$
Zero-copy. Pure Rust async proxy. No JVM, no Python, no overhead.
$
Semantic cache. BGE-small-en-v1.5 runs locally on CPU via Candle. Cosine similarity match across conversation buckets. No external embedding API.
$
Dynamic routing. Cheap model for simple prompts. Premium model for complex reasoning.
$
Single binary. Deploy on any x86-64 Linux VPS. No dependencies.

Download v0.2.2

Pre-built binaries for Linux x86_64 and Windows. Drop it in, change your base_url, done.

↓ GitHub Releases Email for support