@eladhoffer
Joined July 2014
  • Tweets32
  • Following72
  • Followers39
7Media
RAG and long context are usually treated as rival approaches, but both pick the evidence that matters, just at different scales. What if one LLM did both? UNREAL: one frozen LLM retrieves from a 3B-token corpus and filters its own 128K prompt. arxiv.org/abs/2610.08463
2
6
1
15
569
more replies
💡 Takeaway: evidence-sparse long-context tasks and retrieval are one problem at different scales, and a single model can solve both. Bigger context windows aren't enough. Models should be trained to recall.
1
2
34
Joint work with @qyndrmn, @YochaiBlau, @chmiel_brian, @RonBanner1, @soudry_daniel and Boris Ginsburg @nvidia
2
37
Accepted to NeurIPS 2026!
Elad Hoffer@eladhofferMay 8
Excited to share our new arXiv preprint: "Retrieval from Within: An Intrinsic Capability of Attention-Based Models" We introduce INTRA, a framework where attention-based models retrieve from their own internal representations. arxiv.org/abs/2605.05806 1/5 🧵

Retrieval from Within: An Intrinsic Capability of Attention-Based Models

Retrieval-augmented generation (RAG) typically treats retrieval and generation as separate systems. We ask whether an attention-based encoder-decoder can instead retrieve directly from its own...

arxiv.org
5
154
Elad Hoffer retweeted
Nirit Nussbaum@NiritNussbaumJun 7
Thrilled to share our new paper: "LLM Explainability with Counterfactual Chains and Causal Graphs"! 🚀 We introduce a fully automated, model-driven method to extract global, concept-level causal graphs of an LLM's internal reasoning. 📄 arxiv.org/abs/2606.05972 🧵👇 [1/8]
2
12
68
5,362
Excited to share our new arXiv preprint: "Retrieval from Within: An Intrinsic Capability of Attention-Based Models" We introduce INTRA, a framework where attention-based models retrieve from their own internal representations. arxiv.org/abs/2605.05806 1/5 🧵

Retrieval from Within: An Intrinsic Capability of Attention-Based Models

Retrieval-augmented generation (RAG) typically treats retrieval and generation as separate systems. We ask whether an attention-based encoder-decoder can instead retrieve directly from its own...

arxiv.org
1
6
3
24
7,135
more replies
On QA benchmarks, INTRA improves both complete-evidence recall and end-to-end answer quality, especially on multi-hop settings where assembling the right evidence matters. It also amortizes context encoding by reusing precomputed encoder states. 4/5 🧵
1
2
189
Huge thanks to my wonderful co-authors at Nvidia: Yochai Blau, Edan Kinderman, Ron Banner, Daniel Soudry, and Boris Ginsburg. Looking forward to feedback, questions, and discussions! Paper: arxiv.org/abs/2605.05806 5/5 🧵

Retrieval from Within: An Intrinsic Capability of Attention-Based Models

Retrieval-augmented generation (RAG) typically treats retrieval and generation as separate systems. We ask whether an attention-based encoder-decoder can instead retrieve directly from its own...

arxiv.org
1
147
Data Movement Is All You Need: A Case Study on Optimizing Transformers by Andrei Ivanov (ETH Zurich); Nikoli Dryden (ETH Zurich)*; Tal Ben-Nun (ETH Zurich); Shigang Li (ETH Zurich); Torsten Hoefler (ETH Zürich)
1
9
21
Our new work on training convnets using random image sizes: arxiv.org/abs/1908.08986

Mix & Match: training convnets with mixed image sizes for...

Convolutional neural networks (CNNs) are commonly trained using a fixed spatial image size predetermined for a given model. Although trained on images of aspecific size, it is well established...

arxiv.org
2
techcrunch.com/2019/06/17/ha…

Habana Labs launches its Gaudi AI training processor | TechCrunch

Habana Labs, a Tel Aviv-based AI processor startup, today announced its Gaudi AI training processor, which promises to easily beat GPU-based systems by a

techcrunch.com
1
Habana Labs bursts onto the scene with a dedicated AI processor that smokes the competition. eetimes.com/document.asp?doc…

Startup's AI Chip Beats GPU - EE Times

Startup Habana is demonstrating a deep-learning accelerator targeting data centers that outperforms Nvidia’s Volta C100 in inference jobs.

eetimes.com
5
7
Two papers accepted to @NipsConference ! - "Norm matters: efficient and accurate normalization schemes in deep networks" as spotlight - "Scalable Methods for 8-bit Training of Neural Networks" as poster
4
Our paper "Train longer, generalize better" got accepted to oral presentation at #nips2017 ! Arxiv preprint: arxiv.org/abs/1705.08741
3
9
Load more