We revisited Independent Component Analysis (ICA), a decades-old statistical method, and found that it works far better for interpreting LLM activations than we expected! This old baseline deserves a second look. Across GPT-2 Small, Gemma 2 2B, and Qwen 3.5 9B Base, we find ICA is actually competitive with the public SAEs across interpretability, sparse reconstruction, concept prediction, and targeted probe perturbation, while fitting in minutes per layer and using 7–43× less parameter memory. We're releasing ICA Lens, which lets you quickly discover, annotate, and evaluate interpretable directions in LLM residual-stream activations. No need to train another dictionary! More broadly, our results suggest that non-Gaussianity is a useful inductive bias for efficiently finding interpretable directions. Try it: pip install icalens pypi.org/project/icalens/ Paper: arxiv.org/abs/2606.11722v2 1/8