Ben Geist (@b_geist) header
Ben Geist (@b_geist) profile picture

Senior Research Eng @ramplabs / physics + math nerd / Kate Bush fan

Brooklyn, NY
Joined July 2019
Photo by Ben Geist (@b_geist)Photo by Ben Geist (@b_geist)Photo by Ben Geist (@b_geist)Photo by Ben Geist (@b_geist)Photo by Ben Geist (@b_geist)Photo by Ben Geist (@b_geist)Photo by Ben Geist (@b_geist)Photo by Ben Geist (@b_geist)Photo by Ben Geist (@b_geist)Photo by Ben Geist (@b_geist)
We just released an oss version of PorTAL, including training scripts and links to the paper’s models on Hugging Face: github.com/ramp-public/porta… Looking for alpha testers and feedback, give it a spin and let us know what you think!
5
6
1
63
8,468
How do you take a Waymo and not get excited for the future
3
38
1,954
MelatoNaN
opened ig to find tinashe with our old @modal OOH
1
15
1,955
Sparse attn should be synthesis of context instead of selection of context
1
1
4
316
Bjork Hunter x Roman Synaulia
5
299
Average bjork beat on homogenic
Ancient Roman music is the most unsettling shit I’ve ever heard in my life
Readers added context they thought people might want to know
The music is not from a surviving score. It is new music based on guesswork. en.wikipedia.org/wiki/Synaulia
1
6
483
and on the 8th day god made peanut m&m’s
4
19
1,266
many are saying this
Memory is what allows Claude to learn from experience. In a new Engineering Blog post, we share how “dreaming” helps Claude organize what it’s learned, correct its mistakes, and find what it needs more easily the next time. anthropic.com/research/compo…
1
66
31,028
Ben Geist retweeted
🦍 Introducing Memorilla: memory as a modality for LLMs Memorilla turns everything a frozen LLM needs to remember into just 16 memory embeddings ⚡ It uses up to 4,212× fewer tokens and is 27× faster than full context inference while more accurate 🎯 Pushes the Pareto frontier over RAG and agentic memory in personalization, long-document and knowledge QA A thread 🧵:
62
103
26
1,009
101,136
Humans form short-term memories throughout the day and consolidate them into long-term memory while we sleep. Dreaming plays a huge role in this process, replaying details from the day in different contexts and scenarios. I think LLMs should have a similar process for long-term memory consolidation. In this setting, dreaming is self-study. We create a self play game that identifies gaps in existing knowledge and can generate targeted experiences that strengthen memory in those areas. Please read more about our work on this below
11
3
84
6,585
Ben Geist retweeted
Reframing cartridge training as a self-play game cuts the training steps needed by more than half, just by asking better questions. This post continues our series on cartridges, compressed KV caches of long documents. Here we focus on self-study, the method used to train them. Self-study teaches a model reading the cartridge to answer the way it would with the full document in context. Currently, its questions come from fixed, generic prompts, so many of them ask about things the cartridge already knows.
8
9
1
130
7,262
Will be in SF Nov 1-22 🌉
7
28
1,834
About once a month, when a member of @RampLabs finishes their research, I get to moonlight as a designer. This is often my favorite day of the month. Here are some of the visuals I have created in the process:
7
1
78
4,103
When I start pulling out this chart you know it’s getting serious
1
9
661
Ben Geist retweeted
Recent KV cache compression research has focused on small models with dense attention. We scaled these techniques to GLM 5.3, a frontier sparse attention model. We adapted KVzip and Attention Matching, reducing KV cache size by 80% while retaining over 90% of full-context accuracy on the QuALITY benchmark.
9
22
3
202
22,593
RL for continuous learning is slow and can lead to catastrophic forgetting; OPSD does not work reliably. At Ramp Labs we’ve been taking an alternative bet to continuous learning, one that involves efficient accumulated memory in the KV cache over changing parametric knowledge. This blog is the start of a multi part series on techniques my coworker Jeff and I have been exploring over the last few months to bring these techniques to the frontier. Watch this space 👀
6
11
2
191
21,519
Ben Geist retweeted
Readers added context they thought people might want to know
The video is an animation by artist R.J. (@RJ16848519) from his Screen Time series, shared here without credit. x.com/rektober/statu… x.com/connoissurreal… x.com/kikanicolela/s…
363
22,771
1,185
159,494
5,177,511
Ben Geist retweeted
there are many many people who studied physics and came to the conclusion that working on machine learning would probably create more scientific progress than studying physics including many of the original chatgpt crew like john schulman, liam fedus, amodei & jared kaplan at ant
169
189
45
4,399
254,672
Ben Geist retweeted
We had early access to Opus 5.5 and ran it on Ramp Accounting Bench. It performs close to Fable 5.1, at 61% lower cost and 1.7x faster.
4
9
1
183
20,337