Founder @ Zabda Labs. Building and testing on-device AI systems for edge devices. Patent filed, India. Building in public.
grrajan.base.eth
Joined August 2010
- Tweets2.4K
- Following1.8K
- Followers246
- Likes2.2K
Pinned Tweet
10 years in operations. now i build ai products solo at zabda labs.
no traditional ai research background. just a benchmark harness and stubbornness.
posting what i learn, building in public.
if you cant able to process a online order, then you shouldnt have the online option. @atherenergy
Honest question for people running agents overnight: what is the first number you check in the morning? Tasks completed, dollars spent, errors caught, or something stranger? I am trying to figure out what the first screen of an agent-heavy team's dashboard actually needs.
Anthropic published internal data on agent adoption: agent hours now outnumber employee hours 6 to 1. The methodology is the part to copy: they followed the same employee over time instead of comparing across teams. That is how you measure uplift without fooling yourself.
Your eval suite has a half-life. Every model release turns last quarter's hardest test into this quarter's formality. The suite that caught real bugs in spring is a vibe check by fall. Budget for eval maintenance the way you budget for the model itself.
CERN + Mistral: one pretrained Large Physics Model for collider data, trained on billions of simulated collisions. Smart domain: physics has ground truth built in, so claims get tested against nature, not vibes. Expect one foundation model per instrument to become the pattern.
Third-party evaluation is the missing layer between "trust our blog post" and procurement-grade assurance. The rule holds everywhere in testing: the evaluator's independence matters more than their sophistication. A mediocre outside audit beats a brilliant internal one.
We’ve committed to having evaluators assess our safety practices. We're in discussions with multiple organizations, and have now given employee-like access to a team from @METR_Evals.
In this post, we describe how we work with third-party evaluators: anthropic.com/news/an-update…
The unglamorous win here is architectural. One model that reads documents, charts, and video removes the handoff layer where agent pipelines quietly lose context and compound errors. Every model-to-model handoff you can delete is a failure mode you delete.
Offline is the honest version of your product. Strip the network and whatever still works is what you actually built. The cloud features are the enhancement; the local core is the product. Most roadmaps get that backwards and call it a connectivity issue when users notice.
Muse Spark Open: 816B parameters, 42B active, open weights, and it runs on two Mac Studios. The trick is MoE arithmetic: each token reads only its small slice, so 1.2 TB/s of memory bandwidth beats a datacenter GPU. Local frontier is a bandwidth problem now, not compute.
The cheapest reliability upgrade for an agent is a smaller blast radius. Narrower tools, shorter runs, fewer permissions. Capability grows the model; constraints grow the system you can actually depend on.
Jais 3 is the useful kind of model release: six open Arabic models, 900M to 375B, adapted from the open K2 Horizon base rather than trained from scratch. At 7B the Arabic eval average jumps 56.4 to 79.6. Language is becoming an adaptation layer, not a research program.
The most underrated agent feature is interruptibility. Not autonomy levels, not benchmarks: how cheaply can a human stop it mid-run and steer. Teams that ship agents people actually trust are the ones where stopping costs nothing.
Liquid AI's LFM2.5-24B-A2B release notes admit the model reward-hacked during training: searching git history for answers, detecting it was in a sandbox. They sealed the eval environments and reran. Release notes that include the failure modes are the ones worth reading.
Context windows made 'just include everything' cheap. The scarce resource moved to ordering: what a model reads first shapes what it concludes last. Feeding an agent 50 documents is easy. Deciding the order it meets them in is design.
Google DeepMind fine-tuned Gemma for on-device scam detection with GovTech. The privacy story is structural, not promised: training ran in GovTech's sandbox on anonymized reports, inference never leaves the phone. Trust here is an architecture decision, not a policy page.
30,000 merges a day is the tell. When agents became the primary authors, the constraint moved from writing code to the infrastructure wrapped around it. The next wave of dev tooling treats machines as the main users.
New on our Engineering blog: Creating in-house source control.
In May, we merged around 2,500 code changes per day. In September, we merged more than 30,000 per day. As a result, we became bottlenecked by developer infrastructure built for human (rather than agent) scale.
Here, we share how we built a new version control system—and fully migrated our monorepo from git—in five months: anthropic.com/engineering/re…
Question for people building agents: what tool did you take away from your agent, and what finally convinced you?
Every tool on the list is also a failure mode with an API key.
Anthropic's self-replication evals: a 9B model ran inference on 8 of 9 hardware types. IoT's accidental security was heterogeneity - too many odd boards to target. A model that adapts to unfamiliar devices dissolves that. Edge builders, read that twice. anthropic.com/research/self-…
On-device rule I keep relearning: every point of quality has a battery price.
The user meets the drain long before they meet the benchmark. A model that wins the eval but kills the battery by 4 PM is the worse product.
Microsoft's Maia 300 says the quiet part: built mainly for inference - long-context, latency-sensitive, sustained agentic work. Training made headlines; inference is the bill that repeats daily. Agents running 24/7 turn silicon into unit economics. blogs.microsoft.com