에그 retweeted
Best wishes for everyone at Meta for their H1 PSC/checkins. And sorry that the snack have not improved.
에그 retweeted
On "Why would I pay for SaaS if I can vibe-code it?". Here's my 🌶️ take:
1) Headcount
You don't need as many engineers as you had. You can do with less. But the reality is that you could already do with less even before AI. That's management honesty.
1/4
에그 retweeted
Congrats to the ARC Prize 2025 winners!
The Grand Prize remains unclaimed, but nevertheless 2025 saw remarkable progress on LLM-driven refinement loops, both with "local" models and with commercial frontier models.
We also saw the rise of zero-pretraining DL approaches like HRM and TRM. Lots of new learnings!
에그 retweeted
So is the dataset any good? We trained Qwen-3 Instruct and Thinking models on our dataset and used DABstep benchmark to validate the dataset impact on model’s EDA skills. Jupyter Agent Dataset can give up to 22% increase in easy score!
에그 retweeted
Agents as scientific assistants:
- Imo a super promising field to look at, and 2 benchs are very cool to test this capability: SciCode, more on the code side and DABStep, on the data analysis side
에그 retweeted
🧠 Cost analysis of OpenAI’s new models using DABstep baseline:
GPT-4.1 delivers similar performance to o3-mini with just a few points difference - at half the cost. 💸
Might be the sweet spot for budget-friendly agentic tasks. 🤖📉
에그 retweeted
We tested 4 latest models on DABStep benchmark 📊
• Claude 3.7 Sonnet
• Gemini 2.5 Pro
• Llama 4 Maverick
• Llama 4 Scout
Key findings:
• Claude dominates agentic coding tasks 🤩
• Open-source models matching GPT4o for agents 🔥
Full analysis in thread
에그 retweeted
Can state of the art AI LLM models solve today real world data analysis problems?
Well unfortunately it seems like no.
DABStep: A new extremely challenging benchmark for model training teams
Introducing DABStep: Data Agent Benchmark for multi-step reasoning.
We teamed up with @Adyen to test if current LLMs can solve *hard* and *real-world* data analysis tasks.
TL;DR: No! They often fail to read the manual or debug errors. The best model only gets 16% right!
에그 retweeted
(4/13) 🤖 Synthetic Data GenerationWe’re replicating DeepSeek-R1’s synthetic reasoning dataset, experimenting with 32x H100 GPUs and optimized vLLM inference for efficiency.
에그 retweeted
Has anyone figured out how to convert o1/r1 into an agent? We're finding that the classic ReAct prompt produces garbage results 🤔
에그 retweeted
Welcome to @Adyen as a new @huggingface enterprise hub customer (huggingface.co/enterprise). They also published a very interesting post about how they use TGI (our fully open-source inference solution): huggingface.co/blog/martinig…. Cheers!
에그 retweeted
Stateful vs Stateless training? I heard most (?) companies stick with stateless training to avoid issues like catastrophic forgetting. Interesting counter example by Alex Egg (arxiv.org/abs/2107.07106).
At Grubhub, stateful training resulted in 45x savings & +20% metrics increase
에그 retweeted
I propose that we adopt the term "Large Self-Supervised Models (LSSMs)" as a replacement for "Foundation Models" and "LLMs". "LLMs" don't capture non-linguistic data and "Foundation Models" is too grandiose. Thoughts? @percyliang
에그 retweeted
And now we can buy and sell prompts for LLMs, DALLE, and more! 😁😁 It was only a matter of time...
promptbase.com/
