Graduate student in Neurobiology & Behavior at Columbia University • Munich, Oxford, New York • Neuroscience, statistics, mathematics • He/him
New York
Joined March 2017
- Tweets148
- Following689
- Followers307
- Likes343
Pinned Tweet
New preprint, w/ @neuro_kim! We characterize the compositional generalization behavior of kernel models. Our theory derives a new compositional generalization class, highlights key failure modes, and is empirically valid for deep neural networks. (1/24)
arxiv.org/abs/2405.16391
I’ll be presenting my joint work with @Jack_W_Lindsey on the inductive biases of finetuning and multi-task learning at @NeurIPSConf this Friday. Since the paper has changed substantially over the last year, it’s time for an updated tweeprint! (1/13)
neurips.cc/virtual/2024/post…
Both of these regimes generalized worse, providing initial evidence that the nested feature selection regime may have practical relevance. (12/13)
Lots more details on this are in the paper! We’re very excited about this direction and are actively working on follow-ups. If you’re curious about any of this, please come check out the poster on Friday (or reach out to us in any other way :) ). (13/13)
openreview.net/pdf?id=UwvjJZ…
Samuel Lippl retweeted
When do transformers length-generalize?
Generalizing to sequences longer than seen during training is a key challenge for transformers. Some tasks see success, others fail — but *why*? We introduce a theoretical framework to understand and predict length generalization.
Samuel Lippl retweeted
How does the brain bind action to value?
How does it navigate a tradeoff between this binding and generalization to novel situations?
Out today in Nature Neuro! rdcu.be/dUbMd
@JustFineNeuro @NeuroPolarbear @BecketEbs @neuromochi
See original 🧵 + updates below
How does the brain represent multiple different things at once in a single population of neurons? @JustFineNeuro, @NeuroPolarbear, @BecketEbs, @neuromochi and I show that it uses semi-orthogonal subspaces for each item.
Preprint here: arxiv.org/abs/2309.07766
Tweets below! (1/n)
Samuel Lippl retweeted
We live with the limitations of our memory, but don’t really know where they come from. Our new paper (doi.org/10.1073/pnas.2401032…) studies "swap errors", which we argue arise during memory manipulation – see thread for more! @timbuschman, @MattPanichello, @wjeffjohnston
Excited to announce (a bit belatedly) that this is now out @PNASNews. If you're interested in how standard statistical learning principles can enable relational generalization, check out our article here: pnas.org/doi/abs/10.1073/pna…
How do learning systems implement relational and compositional generalizations? In our new preprint (w/ Kenny Kay, Greg Jensen, @vferrera, and Larry Abbott), we investigated this question using transitive inference as a case study. (1/13)
biorxiv.org/content/10.1101/…
Taken together, conjunction-wise additivity may be a useful compositional generalization class: on additive tasks, analyzing kernel models can help us understand behavior in deep networks; on non-additive tasks we need to investigate other learning mechanisms. (23/24)
ALT Example tasks, sorted according to whether they are additive or non-additive. This illustrates that our generalization class cuts across a range of alternative categories of compositional generalization.
Lots more details on this are in the preprint. We’d love to hear any thoughts on this work and are excited about exploring these phenomena further in future work! (24/24)
arxiv.org/abs/2405.16391