Computer scientist obsessed with markets, Founder builder @the_nof1
Manhattan, NY
Joined February 2022
- Tweets205
- Following1.1K
- Followers1.7K
- Likes884
Pinned Tweet
1/6 🧵
How do you train a language model to become a self-adaptive, long-horizon optimization engine for executable trading policy search?
In our new paper, "The Time Value of Evolution," we introduce Lineage-Value Policy Gradients, or LVPG, a new long-horizon RL technique for adaptive evolutionary search.
Matthew Siper retweeted
New episode of The Information Bottleneck podcast featuring... me! (Wait, does this mean that I'm information or a bottleneck?) Anyway, we had a great conversation about games and AI, lots of laughs, and, I think, a good amount of information as well.
New episode of The Information Bottleneck with Julian Togelius @togelius about AI and games 🎮🕹️🎲
Julian (NYU professor, co-founder of modl.ai) has been working on AI for video games since 2004, back when people told him it wasn’t real research. Basically every kid’s dream job 😍
A few things from our conversation:
- Studios use way more ML than they admit. Julian’s advice for finding out what actually ships: go to GDC afterparties (The goalkeeper in EA FC is an RL-trained network, apparently.)
- Games are designed not to need much AI. Dialogue trees are awkward, but players know what to expect and designers know how to work with them. Put an LLM in Skyrim, ask an NPC how much a hamburger costs, and things get weird.
- About AGI: can we learn any game it’s never seen as quickly as a human? Making sure it’s never seen the game is the tricky part. You’d basically need to put a game studio at the South Pole and air-gap it.
- He also thinks we invented the wrong AI. We spent years building machines that copy us, when we could have built something that helps us do the things we’re bad at.
Super fun conversation. I'm sure you will love it
The episode - the-information-bottleneck.c…
This is an absolute game changer
I've always believed that intelligence should come from simple, general building blocks that learn their own abstractions directly from raw data; in other words, the bitter lesson. I think we're finally seeing a real step towards AI that can learn from all data.
Language modeling on raw bytes is a microcosm of this ideal, and to me it's a central problem in AI: if we can't build LLMs that learn end-to-end from bytes, how will we handle the rest of the world's data that we understand even less?
Last year we introduced hierarchical networks (H-Nets) as a first step. Our new version finds up to 3-5x efficiency gains in controlled scaling law experiments, and at larger scale matches an open-source model trained on 11T tokens with ~10x less data and compute. The posts below walk through what we found and why we think it works.
Matthew Siper retweeted
.@Amidos2006 presenting our work (led by @MatthewSiper ) on Procedural Content Metageneration via Continual Abstraction Discovery here at @ieee_cog
Matthew Siper retweeted
This week on The Information Bottleneck we're talking with Julian Togelius (@togelius ) 🥳🥳🥳
Julian directs the Game Innovation Lab at NYU and co-founded modl.ai. For years he's worked on AI and games from both directions: using AI to generate game content and test games, and using games to actually measure what AI can do. Lately he's been digging into why LLMs that write code so well still fail badly at playing games, and what that gap tells us about spatial reasoning and learning in current models.
We'll get into game playing as a benchmark, procedural content generation, open-endedness, and where he thinks the AGI conversation goes wrong.
What should we ask him? Drop your questions in the comments
🧵 1/5
What happens when a language model doesn't just write code, but dynamically creates its own expert API mid-run?
Instead of evolving individual game levels, our new paper evolves entire generator programs using an LLM as the mutation and crossover operator.
4/5
CAD raised mean final-best in all comparisons across Sokoban, Zelda, Lode Runner, and Dangerous Dave.
The endpoint favored CAD whether search began with no helper library or a fixed hand-written expert API.
5/5
CAD turns program search into a process whose vocabulary changes during the run. Later programs inherit callable abstractions discovered from earlier high-fitness programs.
Accepted at IEEE CoG 2026.
With @Amidos2006 @togelius.
Website: matt-quant-heads-io.github.i…
Paper: arxiv.org/pdf/2608.17947
Matthew Siper retweeted
🚨 new preprint: Information Abundance Paradox
TL;DR: Longer context can weaken parametric learning in LLMs.
We propose the Information Abundance Paradox: when relevant information is abundant in the training context, the model has less incentive to internalize that information in its parameters.
Matthew Siper retweeted
At Nof1 we believe the next breakthrough in AI after reasoning is adaptation
If reasoning is the ability to detect patterns and leverage them, adaptation is anticipating how those patterns will change
This is a crucial capability for real world AI, and current LLMs struggle with it. We've run thousands of live market experiments and no amount of context, prompting, or harness engineering has worked. So, we've been pushing our research deeper down the stack
In this paper, we trained models that can better adapt in dynamic environments like markets. They learn to value an action by the futures it opens up, not just its immediate payoff
It's an early step, but we're building towards forever learners that are default forward-looking, rather than static and backward looking
1/6 🧵
How do you train a language model to become a self-adaptive, long-horizon optimization engine for executable trading policy search?
In our new paper, "The Time Value of Evolution," we introduce Lineage-Value Policy Gradients, or LVPG, a new long-horizon RL technique for adaptive evolutionary search.
5/6
Stronger finite-budget search:
Across 90 matched paired runs evolving quantitative trading policies, LVPG increased mean sealed-test Sharpe from 0.86 to 1.32, while peak sealed-test Sharpe reached 4.13.
6/6
By reframing evolutionary search as long-horizon RL, we train a language model to self-correct and autonomously evolve robust policies.
This work was done with my co-authors @togelius @Amidos2006.
Preprint: arxiv.org/abs/2608.13297
#ReinforcementLearning #EvolutionaryComputation #LLMs #MachineLearning #AlgorithmicTrading
1/9
What if natural language were not just a prompt, but the genome of an evolutionary system?
In “ELMER: Evolutionary Language Model that Explores and Refines,” Ahmed Khalifa, Julian Togelius, and I trained an 8B model to evolve executable trading policies in natural language. 🧵
8/9
Requested mutation strengths systematically alter the composition of semantic edits, while language mutations preserve more parent fitness at matched small-to-moderate behavioral displacements.
9/9
Grateful to my coauthors Ahmed Khalifa and Julian Togelius for building this work with me.
Paper: arxiv.org/abs/2608.10196
Code: Coming soon
#EvolutionaryComputation #LargeLanguageModels #ProgramSynthesis #AlgorithmicTrading
