🧵0/7 🚨 Spotlight @ICML2025 🚨 Chi-Ning and Hang have been thinking deeply about how feature learning reshapes neural manifolds, and what that tells us about generalization and inductive bias in brains and machines. They put together the thread below, which I’m sharing on their behalf 👇Enjoy!

NYU-CDS article about MMCR (Maximum Manifold Capacity Representations): The key principle of MMCR is to maximize the number of image manifolds (generated by nuisance variations) that can be linearly decoded in the representations, hence maximizing the efficiency of the representation from the perspective of a decoder ('task-centric' efficient coding, or efficient 'decoding', but groups of responses defining a 'manifold' are variations of the same image, so it also has to do with efficiently representing stimuli). The original idea of 'Manifold Capacity' comes from statistical mechanical analysis of the capacity of perceptrons, generalized from discrete points to 'neural manifolds'. very nice to see the interaction between different fields: from a new theory (merging statistical physics, high-dim geometry and convex optimization) to a new SSL algorithm, a new brain model, and new theoretical analyses. Check out the information-theoretic analysis of MMCR: arxiv.org/abs/2406.09366 Original MMCR paper: openreview.net/pdf?id=og9V7N… Original paper on Manifold Capacity: journals.aps.org/prx/pdf/10.…

🔥Lots of new theories on day 3 of Cosyne: [3-100] capacity for nonlinear classification of manifolds [3-105] theory of multitask learning (optimal repr geometry + geometric measures for data analysis) [3-167] tuning diversity shapes efficient representation geometry 🧵👇🏻

👋Check out the latest findings from our lab at #COSYNE24 @CosyneMeeting Thread & links to preprints to follow (1/n)🧵👇

Just arrived in New Orleans for #NeurIPS2023 this week. Topics I am excited about: neuro-AI, neural manifolds (representation geometry), stat physics for machine learning, interpretability, relational & causal representations My group is presenting their awesome work👇 (1/n)

A great piece by @SimonsFdn about our theory on the capacity of neural manifolds with correlated variability, led by @AlbertWakhloo. Many thanks Mara Johnson-Groh & @LucyIkkanda.