Krea 3 by @krea_ai is live on @useRouterPlus great to have partnered with the team for day 0 support would love to see examples of what you folks do with it on RouterPlus !! What models would you like us to see support next ?

Self hosting has inference providers alternative now. So unless you have reasons to own hardware and self host or you are post training, everyone doesnt need to download anymore. You can access these services like subscription. Routers make switching, discovery and experimentation far more convenient than you downloading and trying. Routers are also catalog now, an active one and not a passive listing. People now go to marketplaces like @openrouter, @vercel , @useRouterPlus for their discovery vs huggingface h/t to @ParikhRajesh07

btw at @useRouterPlus we have been using numerous techniques to ensure that inference providers are ACTUALLY delivering what they claim - this involves running relatively expensive live evals , checking quality and also measuring symmetric KL divergence to monitor deviations from actual expected behaviour since a lot of inference contracts go through us we can actively monitor for such discrepencies and hold providers accountable. its only done now for certain larger contracts - but we expect to roll it out to the wider platform soon

Buying inference today means picking between two bad options. Commit for a year and pay for capacity you don't use. Or go serverless and get rate limited the moment you grow. But usage arrives in bursts: a weekend eval, a two-week backfill, US business hours. And not all tokens are equal. You might need BF16, not quietly quantized FP8. Served in the US. A latency target. Zero retention. Fast tokens for your product, cheap batched tokens for the rest. The capacity exists. It's just scattered across big clouds and long-tail providers you've never heard of. So we built a marketplace. Introducing the @useRouterPlus managed inference marketplace. Post the inference you need, when you need them. Commitment you are comfortable with. Providers bid. You pick the one that meets your spec. we're already on track for 1 trillion tokens a month. routerplus.com/capacity

we are starting to host small gatherings around AI and inference with the first of the series tonight at 6:30pm at Hogpatch ( pizzas and snacks included ! ). Great chance to hang out with folks thinking and building in the space. luma link below ps : we will be rebranding to @userouterplus, would love to show you what we’re upto as well!

inference engg chart porn inside @EVO__HQ dashboards cc @XLN1999

deployed and served custom models for a big @evo__hq customer over the past week insane and rapid learnings understood their traffic patterns, setup evals - did lot of ablation runs on multiple models eventually we weren't getting enough savings, so we posttrained a model - first SFT and then other RL policies then came inference optimization setup end to end , including custom spec dec models as per their data distribution, tuning vLLM configs, quantization etc then more work around compute allocation , inference capacity planning / warming up as per traffic profiles etc all orchestrated by our in house autoresearch / AI engineer all this to ensure our customers get best bang for buck we are now rapidly onboarding inference providers who can reliably service our growing demand and if you are someone whose agentic/AI workload spends are more than $20k a month, then do reach out. We would love to figure what we could do for you!

preach ! been telling ya’ll also the most important part is the last line. “the challenge is customizing them to an org/team/individuals requires data” this is exactly what @evo__hq solves for btw

“There is no best model for every task. Routing, harnesses, evals, and the right mix of open and proprietary models can radically change the economics.” exactly the thesis of us at @evo__hq btw :)

Introducing EVO Router - a smart router that continuously optimizes and hill-climbs on your AI inference workloads. It learns from your code, prompts, production traffic, use cases, and SLAs, then searches for the best setup for every workload. That can mean more than choosing just a single model or routing through a classifier. @EVO__HQ can optimize across model × provider combinations, fusion systems, cascades, routing policies, and prompts — whatever the workload allows, while staying within your quality, latency, and reliability constraints. As open-source models move closer to frontier-model performance, we believe this kind of workload-specific hill-climbing will become an important part of how AI systems are deployed and operated. Our early users have seen 30–60% lower inference costs across use cases ranging from complex multi-turn agents to asynchronous batch workloads. EVO Router has also reached the Pareto frontier on several benchmarks, outperforming existing setups on cost, quality, or both.

always great to hear folks making good use of @evo__hq's autoresearch engine T - 48 hours untill we launch the next iteration that the team has been working on, putting to use all the learnings from our users

If you want early access to not just yet another router™, but a system that automatically sets up evals, optimizes your workloads, and continuously finds the best model x provider for YOUR USECASE , then my DMs are open. We already have a few teams in the beta, and they’ve reduced inference costs by 30–60% with no quality regressions. @evo__hq

i am seeing a lot of disillusion amongst folks for the new crop of AI "routers" that are popping up. everyone is in a gold rush to capture demand, w/o deep consideration of the actual problem to be solved i have been talking with a lot of disgruntled users of such products - and the inside feedback from all such customers has been that such tools either end up degrading quality of their product or end up costing more because of non optimal switching and invalidating KV Caches per api call task routers are meaningless when you take into consideration long agentic tasks. $ per mil token input / output is an artifact of the past. Its an exploration - exploitation problem that you have to solve for specific usecases. its something that you need to hillclimb for specific customer on their specific distribution of data - not something a mere global classifier or decision tree will get it right. i would love for folks who have used such product to comment if it really worked for them, or for founders of such "router" products to show what they are doing different will be thinking and posting more on this soon

evo v0.6 is out evo is an opensource autoresearch orchestrator with over 20k+users that turns your coding agent into a structured optimizer: it runs autoresearch experiments against your repo's metric, keeps a tree of what worked, and converges on a real improvement. its dead easy to use with just two simple commands new in v0.6: it supervises its own search, runs on your Slurm cluster, and ships changes into your codebase with taste. details below

new landing page for @evo__hq

weekend @evo__hq activities: given the recent sovereign-AI narrative (amidst anthropic's fable pullback), i wanted to see if evo could help make some of the indian models we have better in whatever way. so i kicked off an autoresearch run on evo to see if @SarvamAI's 30B decode throughput could be improved, at bf16, on a single H100. currently 10+ hours in. so far, evo seems to have found ~3% improvement. the metric is geometric mean tok/s across batch sizes 64 / 128 / 256, measuring steady-state decode only. prefill is timed out, so this is purely per-token decode rate on a fixed workload. evo also ensures that anything that got faster by changing outputs, lowering precision, or messing with MoE routing was rejected by the accuracy gate. the gate compares each candidate against a frozen baseline on both next-token distributions and actual decoded tokens. if argmax agreement or logprob drift moves meaningfully, the change is rejected, even if it is faster. very imp caveat: these are experiment-harness numbers, not production serving numbers. the gain still needs to be validated in a real serving setup before anyone treats it as real capacity. i also havent done an external audit for any benchmark hack the agent may have done. a potential ~3% bump is a potentially a pretty significant improvement for someone like sarvam at that scale. decode is a major part of inference cost. a ~3% decode-side throughput gain at identical accuracy means more capacity, or fewer GPUs, without changing the model. i also want to s/o to @vishnuvig of jarvislabs.ai / @e2enetworks for compute support. i am trying to use more and more compute from indian providers as much as i can and give feedback to improve the experience as well