
Machine Learning @Latent_Labs, anything from classical statistics to modern machine learning and generative AI, previous @altos_labs
United Kingdom
Joined June 2020
- Tweets379
- Following668
- Followers788
- Likes1.7K
Saying there is a "distribution shift" between train and test data is a great marketing ploy in #machinelearning to make overfitting somehow the data's fault, not the model's or modellers'. 🙃
Sebastian Schmon retweeted
We’re excited to announce DiG-bench, a new benchmark for discovery!
Over the last few weeks we’ve been testing frontier AI models on our novel discovery games and seeing how they score. Each game is a text-based environment, so they probe discovery capabilities in the natural domain of language models, rather than requiring additional, potentially confounding, visual understanding.
TL;DR frontier models have improved a lot over the last few months. But they are still stumped by some surprisingly simple problems, even in their native text domain.
With
@cocosci_lab (@Princeton) @MITCoCoSci (@MIT) @SchmidhuberAI (@KAUST_News) @misovalko (@Inria) @tri_dao (@PrincetonCS) @RMBattleday @zebkDotCom @FraserGreenlee @akaijsa @ClareMaguire @TimMuller1 @kubicek_ales @physicscat0x7d @SukritSumant @thoughtchannel_
(1/5)
Sebastian Schmon retweeted
Today we are happy to share one of our first open research releases: Latent Sparse Attention (LSA), a step towards subquadratic attention that matches accuracy of full attention at ultra-long context.
Sparse attention attends to only a few tokens per query, but it has to find the right ones. At 10M tokens, that search takes 98% of the attention layer's time. LSA makes sparse attention up to 22.7× faster and full attention 72.4x faster.
With @kubicek_ales @caciolai @PengLiangzu @SukritSumant @Ben_Hoov @FraserGreenlee @RMBattleday @radim_urban @thoefler @jcrwhittington
I cannot get over the fact that Max is not the maximum.
We’re renaming our Max plans:
Max 5x → Max
Max 20x → Ultra
Max is still $100/month, and Ultra is still $200. If you’re a subscriber, there’s nothing you need to do.
We’re also removing the 5-hour limit for verified Ultra accounts, starting today with most existing subscribers. Newer accounts keep a temporary limit until they’re verified, which helps us prevent fraud and abuse. You'll get an email when it's removed.
Weekly limits remain unchanged. View our plans here: claude.com/pricing
1
3
124
I used @claudeai for orchestration, @bfl_ai for designs drafts, @ChatGPT for blender models, @ElevenLabs for voices and sounds. Implemented using @godotengine
2
60
The extent to which you can just do things now is staggering: I used AI to generate my own RTS, including the possibility to play against an agent by creating a machine readable gaming engine.
I am still wrapping my head around how this will change industries.
1
5
186
Replying to @GuangyuRobert
I have also interviewed with a big bank once and they told me that they could only hire me on an entry level because for them only “years since leaving academia” counts. I was an assistant professor, but for them university was university, whether student or professor. I ended the process then and there.
14
Replying to @GuangyuRobert
It can vary a lot based on who’s in charge. I have also seen some universities in the UK being better and promoting more based on merit. That’s often the less prominent ones who have to find alternative ways to retain talent.
47
One thing that @OpenAI could do with the massive compute is to brute force re-review every research papers, beginning with maths and physics and continue to stats and ML.
4
204
Replying to @GuangyuRobert
Absolutely agree but to be fair it’s the same for most institutions and large companies: if you can’t measure impact or contribution easily on an individual level (either because the signal is noisy or measurement is costly) you divert to poor but cheap proxies.
1
1
556
Claude has written me some of the best explanations on topics that were absolutely worth reading and there is a lot of human generated text on the internet that is factually wrong or at least badly written.
52
Unpopular(?) opinion: we don’t need classifiers to detect whether something is LLM generated or not, but rather whether it’s worth reading, i.e. we need to detect slop vs non-slop, not AI vs human.
Those are very correlated but not identical.
4
10
245
Replying to @SeBayesian @andrewgwils
You can potentially desk reject simply by looking at bad claims or claim that cannot be tested with the selected instrument of validation.
36
Replying to @andrewgwils
Have one field for “claim”, with low number of maximal characters and another field for “how the claim is validated” (could be a benchmark or proof). Precise high level summaries will help guide reviewers and not get people confused in LLM verbiage.
1
4
304
Replying to @andrewgwils
Reduce maximum page count. PRL (most prestigious physics journal) can get away with 4 pages. Being succinct forces being very deliberate and concise in contribution and gives humans at least a chance to get a sense of the idea without having to invest in pages of AI slop.
5
341





