Sunjay Dhama

here is my meticulously curated (and highly biased) summer paper reading list 📚: Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks (2020) ╰╴ arxiv.org/abs/2005.11401 LoRA: Low-Rank Adaptation of Large Language Models (2021) ╰╴ arxiv.org/abs/2106.09685 Ring Attention with Blockwise Transformers for Near-Infinite Context (2023) ╰╴ arxiv.org/abs/2310.01889 Mamba: Linear-Time Sequence Modeling with Selective State Spaces ╰╴ arxiv.org/abs/2310.01889 The Era of 1-bit LLMs: All Large Language Models are in 1.58 Bits ╰╴ arxiv.org/abs/2402.17764 Quiet-STaR: Language Models Can Teach Themselves to Think Before Speaking ╰╴ arxiv.org/abs/2403.09629 Leave No Context Behind: Efficient Infinite Context Transformers with Infini-attention ╰╴arxiv.org/abs/2404.07143 The Illusion of State in State-Space Models ╰╴arxiv.org/abs/2404.08819 σ-GPTs: A New Approach to Autoregressive Models ╰╴arxiv.org/abs/2404.09562 Rethinking LLM Memorization through the Lens of Adversarial Compression ╰╴arxiv.org/abs/2404.15146 Better & Faster Large Language Models via Multi-token Prediction ╰╴arxiv.org/abs/2404.19737 xLSTM: Extended Long Short-Term Memory ╰╴arxiv.org/abs/2405.04517 Chameleon: Mixed-Modal Early-Fusion Foundation Models ╰╴arxiv.org/abs/2405.09818 Transformers Can Do Arithmetic with the Right Embeddings ╰╴arxiv.org/abs/2405.17399 Scalable MatMul-free Language Modeling ╰╴arxiv.org/abs/2406.02528