
MTS @ParsewaveAI | LLMs • AI Evals • Post-Training
Bhilai
Joined June 2025
- Tweets4.5K
- Following165
- Followers2.2K
- Likes12.2K
Replying to @chongdashu
This might help some of you guys
xitter.cf/initlayers/status/2111…
A quick plug for something we’re doing at OmniRush.
We’re hosting the Frontier Buildathon 2026. If you’ve been wanting to try Astra, or have an idea you keep putting off, come build with us.
It doesn’t have to be your next startup. A useful little tool, an agent, a research experiment, something that fixes a problem you’re tired of dealing with. Just get it working.
Each participant gets 10M free Astra tokens, worth roughly $300, plus access to all GPT-6 models through OmniRush.
There’s also a $300 prize pool for the top two projects and Best Use of OmniRush.
It’s fully online and solo. Projects need to be open source, and you keep full ownership of what you build.
Register by 19 October, 11:59 PM IST. Submit by 20 October, 11:59 PM IST.
Would love to see some familiar faces join. We’ve sorted the compute; choosing which of your 14 unfinished ideas to build is still on you.
A quick plug for something we’re doing at OmniRush.
We’re hosting the Frontier Buildathon 2026. If you’ve been wanting to try Astra, or have an idea you keep putting off, come build with us.
It doesn’t have to be your next startup. A useful little tool, an agent, a research experiment, something that fixes a problem you’re tired of dealing with. Just get it working.
Each participant gets 10M free Astra tokens, worth roughly $300, plus access to all GPT-6 models through OmniRush.
There’s also a $300 prize pool for the top two projects and Best Use of OmniRush.
It’s fully online and solo. Projects need to be open source, and you keep full ownership of what you build.
Register by 19 October, 11:59 PM IST. Submit by 20 October, 11:59 PM IST.
Would love to see some familiar faces join. We’ve sorted the compute; choosing which of your 14 unfinished ideas to build is still on you.
A week ago, if you told me I'd be working with people from OpenClaw and Hugging Face, I'd have laughed.
Funny how much can change in a week.
Replying to @kavya_dua_
I do but it's hard to get enough time when you run 5 parallel agents.
I worked on this audit and learned a lot from going through AutomationBench task by task. The issues were often small but meaningful: a missing field, the wrong record being checked, or a verifier accepting the right words but the wrong result.
Glad we could fix 206 of them and publish the evidence with the release.
AutomationBench Verified is out.
We went through all 600 public tasks in @Zapier's AutomationBench and checked every verifier.
• Agents flagged 323 verifiers as suspicious, human review confirmed 206 real bugs, all 206 are fixed.
• We replayed 1,235 Kimi K3 runs on the old and fixed verifiers and 344 of them (27.9%) got a different grade
• Where verifiers were too strict, pass rate went from 18.8% to 43.8%
• Where they were too lenient, it dropped from 60.2% to 49.7%
Audit and dataset links below.





