Metrum AI (@metrumai) profile picture

Metrum AI. Know Your AI. Grow Your Business.

Joined July 2024
1
2
2
47,875
Metrum AI retweeted
Stop Paying Twice for the Same Tokens: @metrumai shows how to eliminate the KV Cache recomputation tax on the Dell PowerEdge XE9785 with AMD Instinct MI355X, Micron 9550 PRO, and LMCache metrum.ai/blog/kv-cache-reco…
1
2
111
Enterprises are in a bind. Use frontier models and you overrun costs and risk your IP. Don't, and you get disrupted by competitors who did. Or by future releases of the models themselves. Hosted routers close the cost gap, but you give up control. We weren't willing to make that trade, so we built our own. Today we're open-sourcing it. Metrum AI Router: a trainable routing layer that predicts quality and token cost per model and picks the cheapest above your quality floor. It trains on your traffic, on your hardware, and the routing intelligence it produces is yours. Apache-2.0. 85% lower model spend across our own 25 devs on our internal branch. github.com/metrum-ai/router
🤖 Made with AI
1
1
3
37,041
Metrum Insights now supports NVIDIA DGX Spark. Benchmark and evaluate seamlessly. Early results are🔥#SparkSomethingBig @NVIDIAAI
1
1
348
The Metrum AI team had a blast seeing stellar early results on Nvidia DGX Spark and integrating Metrum Insights with DGX Spark to provide a seamless environment for benchmarking and evaluation. #SparkSomethingBig @NVIDIAAI
1
270
Metrum AI retweeted
Overheard at AMD #AdvancingAI: What does an open community mean for AI developers?
1
14
2
202
29,885