Anshul

I'd read these 15 papers before taking this course, they cover EVERYTHING from RLHF to recursively self-improving agents. (optional, but they just make everything click faster imo) links below! 👇 1. InstructGPT - SFT → reward modeling → PPO. - Human preference data + KL regularization. - The full RLHF stack. - Reproduce the pipeline conceptually 2. Deep RL from Human Preferences - Pairwise comparisons. - Reward model training. - Behavior without hand-specified rewards. - The foundation for RLHF. 3. Proximal Policy Optimization Algorithms - Policy gradients and clipped objectives. - Importance sampling ratios. - Advantage estimation. - Reproduce PPO on a small environment. 4. Direct Preference Optimization: Your Language Model is Secretly a Reward Model - Derivation of DPO from RLHF. - Preference-based training without explicit PPO. - Reference policy and implicit reward modeling. - Reproduce DPO. 5. DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models - GRPO algorithm. - Group-relative advantage calculation. - Critic-free RL. - Reproduce GRPO. 6. Let's Verify Step by Step - Process reward models (PRMs). - Step-level vs. outcome-level supervision. - Verifying intermediate reasoning. - Train a small PRM. 7. DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning - RLVR and R1-Zero's emergent reasoning. - Cold-start SFT, rejection sampling, and RL. - Distillation of reasoning models. - Reproduce a small RLVR experiment. 8. DAPO: An Open-Source LLM Reinforcement Learning System at Scale - Improvements to GRPO. - Dynamic sampling and clipping. - Token-level policy optimization. - Large-scale RL engineering. 9. Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters - Inference-time compute allocation. - Best-of-N sampling and search. - Verifier-guided reasoning. - Compute scaling tradeoffs. 10. Tülu 3: Pushing Frontiers in Open Language Model Post-Training - Full open post-training recipe. - SFT, DPO, RLVR. - Dataset construction and evaluation. - How to assemble an end-to-end post-training stack. 11. ReAct: Synergizing Reasoning and Acting in Language Models - Reasoning/action loops. - Tool-use architecture. - Action-observation feedback. - Implement a ReAct agent. 12. Reflexion: Language Agents with Verbal Reinforcement Learning - Episodic memory. - Self-reflection after failures. - Feedback-based improvement without weight updates. - Implement a reflection loop. 13. Voyager: An Open-Ended Embodied Agent with Large Language Models - Skill generation. - Skill libraries. - Ind feedback. - Open-ended learning in an environment. 14. Self-Taught Optimizer (STOP): Recursively Self-Improving Code Generation - Recursive code generation. - Self-modifying optimization systems. - Evaluating whether self-improvement actually works. - Reproduce a bounded self-improvement loop. 15. Darwin Gödel Machine: Open-Ended Evolution of Self-Improving Agents - Agents modifying their own code. - Evolutionary search over agent designs. - Empirical evaluation of self-modifications. - The distinction between agent improvement and model-weight improvement.

no filler, no nonsense: these are the 15 videos that took me from zero GPU knowledge to GEMM optimization research: watch them in this order (links below👇): 1. GPU Architecture Explained by ExplainingComputers Overview: A visual intro to GPU hardware and parallel execution. - GPU versus CPU: throughput-oriented versus latency-oriented design. - Streaming Multiprocessors (SMs) and CUDA cores. - SIMT execution: threads grouped into warps of 32. - GPU memory hierarchy and hardware parallelism. 2. CUDA MODE Lecture 3: Getting Started With CUDA by GPU MODE Overview: A practical intro to CUDA programming and writing GPU kernels. - Host versus device: CPU launches the work, GPU executes it. - Kernel launch syntax and thread indexing. - Mapping data elements to threadIdx, blockIdx, and blockDim. - Basic GPU programming from Python. 3. CUDA MODE Lecture 4: Intro to Compute and Memory Architecture by GPU MODE Overview: A hardware intro to the compute and memory design behind kernel performance. - Compute versus memory: throughput limits versus bandwidth limits. - GPU memory hierarchy: global memory, shared memory, registers, and caches. - Memory bandwidth and compute throughput. - How daexecution time. 4. Memory Coalescing by NVIDIA Developer Overview: A visual intro to how GPU threads access memory efficiently. - Coalesced versus uncoalesced access patterns. - Warp-wide memory transactions: threads in a warp combine their loads. - Strided and contiguous memory layouts. - Transaction efficiency and wasted bandwidth. 5. CUDA MODE Lecture 9: Reductions by GPU MODE Overview: A practical intro to parallel reduction algorithms on the GPU. - Parallel sum and max reduction: combining N values into one in log(N) steps. - Warp-level reductions using shuffle instructions. - Shared-memory reductions and thread synchronization. - Handling non-power-of-two inputs and reduction tails. 6. The Roofline Model by Berkeley HPC Overview: A performance model for finding hardware bottlenecks. - Arithmetic intensity: FLOPs per byte of memory traffic. - Peak compute throughput versus peak memory bandwidth. - Compute-bound versus memory-bound: which roof limits the kernel. - Estimating achievable kernel performance. 7. CUDA Performance Checklist by GPU MODE Overview: A practical checklist for diagnosing and optimizing kernel performance. - Memory coalescing and access patterns. - Occupancy versus register pressure: more registers per thread means fewer resident warps. - Control divergence: threads in a warp taking different branches. - Kernel bottleneck diagnosis. 8. Nsight Compute Introduction by NVIDIA Developer Overview: A practical intro to profiling individual GPU kernels. - Kernel-level profiling and hardware performance metrics. - SM throughput, memory occupancy. - Warp stall reasons: why warps wait instead of issuing instructions. - Roofline analysis and bottleneck identification. 9. Nsight Systems Introduction by NVIDIA Developer Overview: A system-level view of the full CPU-GPU timeline. - CUDA API calls, kernel launches, and memory transfers. - CPU launch overhead and GPU idle gaps. - Stream execution and operation overlap. - Synchs and pipeline analysis. 10. CUDA MODE Lecture 14: Practitioner's Guide to Triton by GPU MODE Overview: A practical intro to writing high-performance kernels in Triton. - Triton versus CUDA: block-level versus thread-level programming. - Program IDs, blocked tensors, and tile-based execution. -Array dimensions. - Tuning block sizes, warps, and memory layouts. 11. Fused Softmax in Triton — GPU programming tutorial channel Overview: Implements softmax efficiently by fusing multiple operations into a single GPU kernel. - Numerically stable softmax using row-wise max subtraction. - Reduction for maximum and normalization denominator. - Fusion of max, exp, sum, and division. - Eliminating intermediate global-memory writes and kernel launches. 12. FlashAttention From Scratch in Triton by GPU programming tutorial channel Overview: A practical intro to building an IO-aware attention kernel. - Tiled attention: splitting Q, K, and V into blocks that fit in SRAM. - Online softmax: running max and sum updated block by block. - HBM versus SRAM: keeping intermediates on-chip to cut memory traffic. - IO complexity and memory-efficient attention. 13. CUDA Streams and Asynchronous Execution by NVIDIA Developer Overview: A practical intro to asynchronous GPU execution and pipeline throughput. - CUDA streams: ordered queues of GPU operations. - Asynchronous memory copies with pinned host memory. - Overlapping computation and data transfer; synchronization, events, and inter-stream dependencies. 14. Hopper TMA / WGMMA Deep Dive by NVIDIA Developer Overview: A hardware intro to Hopper's data movement and Tensor Core mechanisms. - Tensor Memory Accelerator (TMA): hardware-driven multidimensional copies into shared memory. - WGMMA: matrix multiply-accumulate across 4 warps. - Producer-consumer pipelines: load warps versus compute warps. - Asynchronous execution and latency hiding. 15. CUDA MODE Lecture 23: Tensor Cores by GPU MODE Overview: A hardware intro to how Tensor Cores accelerate matrix multiplication. - Tensor Core matrix multiply-accumulate operations. - FP16, BF16, TF32, and lower-precision formats. - Matrix tile layouts and Tensor Core execution. - Precision, throughput, and performance trade-offs.

this week in AI was WAY bigger than you realize here are the top 11 releases that directly affect jobs in kernel engineering, security research, sales, even chemical engineering: 1. Together Compiler (TKCC) AI agents continuously rewrite, test, and optimize GPU inference kernels • up to 2.6× higher tokens/sec vs optimized open-source engine configs • 1.7× throughput over available megakernels (Llama 1B, batch size 1) • 252× higher experiment throughput vs baseline agentic optimization systems • already serving production workloads together.ai/blog/together-co… 2. Scientist World turns research papers into executable environments with reproduced baselines • 1,000 verified environments • frontier AI scientists beat the reproduced human methods in 74% of evaluated improvement/innovation cases • 158% boost in a small model's research ideation after training on these environments arxiv.org/abs/2611.08478 3. SwarmEvolve hundreds of cheap agents share KV caches and search for better programs in latent space, without decoding every idea into text • tested across 21 math discovery and scientific computing problems • hit in a median of 3–8 min the scores other frameworks needed 60 min to reach • Qwen3 8B improved on solutions originally found by AlphaEvolve arxiv.org/abs/2611.08483 4. TCSExam benchmark for theoretical CS research: theorem proving, solver engineering, constructions, open conjectures • 100 expert-reviewed tasks • outputs checked by Lean or executable validators • GPT-6 Astra leads Full at 71.9, Opus 5.5 leads Lite at 72.3 arxiv.org/abs/2611.07962 5. SPAR self-evolving B2B sales agent that treats selling strategies as testable hypotheses, remembers each merchant, and updates its playbook using causal attribution • live A/B test on Alibaba's 1688 platform, 200,000+ merchants • 15.8% conversion lift • 35.8% growth in high-intent merchants arxiv.org/abs/2611.01695 6. OpenAI Linux Kernel Autonomous Pipeline autonomous vulnerability discovery and patch generation for the Linux kernel • 2,308 candidate findings • 1,724 vulnerabilities dynamically reproduced • 948 local privilege-escalation exploits, plus candidate patches for maintainer review • ~$1.2M in API costs openai.com/index/a-thousand-… 7. VFlash visual planning model that predicts points, bounding boxes, and trajectories jointly instead of one coordinate token at a time • lower prediction error than autoregressive decoding across 2D and 3D robotics tasks • ~25× faster 3D point-motion generation (preliminary) • 70.2% avg success across four MolmoSpaces suites vs 62.6% baseline arxiv.org/abs/2611.08449 8. ChemE-as-Code open-source platform letting LLM agents do chemical engineering calcs, process simulation, plant layout, HAZOP, and life-cycle assessment through checkable tools • preliminary ammonia engineering package for $224 in API costs • calculations run outside the LLM, with provenance tracking and automated consistency checks arxiv.org/abs/2611.07947 9. GuidedWAM training-free guidance for driving world action models. it injects differentiable objectives during sampling, so you skip retraining for every new objective • tested on three frozen, public world action models • +1.8 to 3.6 EPDMS on the NAVSIM v2 hard split (avg over 3 seeds) • under 10% extra sampling time per step arxiv.org/abs/2611.07912 10. Wave Forcing long-video generation method that co-designs training, inference, and execution for better temporal consistency and throughput • on Wan2.1-T2V-1.3B, quality comparable to Rolling Forcing with 5 denoising steps • 134.8 end-to-end FPS on 8 H200s (4-step) • up to 2.53× throughput over matched context-parallel baselines (DiT-only tests) arxiv.org/abs/2611.00859 11. Persistent Retrieval for Conversational Agents retrieval subagents that keep their history across multi-turn conversations and hand evidence straight to the main agent • 43.3% fewer agent-model tokens vs skill-based retrieval • 45% lower estimated API costs • 9.1% faster average response time arxiv.org/abs/2611.07899 the pattern this week: AI is getting really good at the optimization loop first. kernels, program search, sales playbooks, retrieval if part of your job is "try something, measure it, try again", that's the part to start learning to automate yourself

you can run real physics simulators on a laptop in 2026! here are the 10 i'd actually use for AI research and world models, ranked by how good they are for experimenting: 1. Newton: robotics physics backed by DeepMind, Disney Research and NVIDIA 2. MuJoCo: best overall, easiest to get running 3. NVIDIA Warp: write your own physics kernels in Python 4. PhiFlow: differentiable fluid sims you can train a neural net through 5. Genesis: multi-physics engine built for embodied AI 6. Taichi: for inventing new physics (granular, elastic, MPM) 7. PhysicsNeMo: PINNs, neural operators, surrogate models 8. DeepXDE: neural nets that solve differential equations 9. JAX MD: differentiable molecular dynamics 10. OpenFOAM: serious engineering CFD, steepest setup the one to install fastest (and the two i'd build research on) are next👇

these 10 papers are what AI for biology the coolest field in 2026 if you work anywhere near biology, this is your reading list. a lot of this work is already being tested in real labs! links to the papers below! 👇 1. Evo 2 (Nature, Mar 2026) • genomic foundation model trained on ~9 trillion DNA base pairs, modeling sequences across all domains of life. • predicts the effects of genetic variation, including clinically relevant variants. AI moves from single proteins to whole-genome patterns. 2. AlphaGenome (Nature, Jan 2026) • ingests up to 1M base pairs and predicts thousands of regulatory signals: gene expression, chromatin accessibility, RNA splicing. • gives a mechanistic read on how a tiny non-coding mutation can cascade into disease. 3. Robin, multi-agent scientific discovery (Nature, 2026) • an agent system that generates hypotheses, mines literature, proposes experiments, interprets results and updates its beliefs. • closest thing yet to AI acting as the scientist. experimentally validated therapeutic candidates in a lab-in-the-loop workflow. 4. AutoDNA (Nature, Nov 2026) • couples AI with automated lab instruments to run biomolecular operations with minimal human intervention. • AI orchestrating physical experiments end to end. this points at labs that run research around the clock. 5. AlphaGenome Atlas (preprint, Sep 2026) • in silico mutagenesis of the entire human genome: predicted effects for billions of variants. • a massive prioritization resource for non-coding variants, potentially surfacing overlooked genetic explanations for rare diseases. 6. Claude uplifting biomolecular modeling (Anthropic, Sep 2026) • Claude optimized 30+ specialized biological AI models, reportedly ~4x faster on average while retaining identical outputs. • AI accelerating the infrastructure other AI uses to do biology. a compounding loop. 7. Pinal (Nature, Oct 2026) • 16B-parameter foundation model that translates natural-language descriptions of protein function into candidate designs. • scientists can state the biological goal in plain language. results are computational so far, so wet-lab validation is the next test. 8. Towards a virtual yeast (Nature, Jul 2026, perspective) • proposes an AI system integrating genetic, metabolic and structural processes to simulate yeast-cell behavior. • a virtual cell lets you test hypotheses computationally before paying for the physical experiment. 9. AI-driven digital organism (Nature Medicine, Aug 2026, perspective) • interconnected foundation models representing biology from molecules to cells to whole organisms. • a roadmap for predicting biological outcomes across every level of complexity. 10. Generalist biological AI (Nature Biotechnology, Mar 2026, review) • surveys systems that jointly model DNA, RNA, proteins and cellular systems as one language of life. • the clearest map of where general-purpose biological foundation models are heading. links to the papers, same order: 1. Evo 2 alphaxiv.org/abs/2603.01234 2. AlphaGenome doi.org/10.1038/s41586-025-1… 3. Robin doi.org/10.1038/s41586-026-1… 4. AutoDNA doi.org/10.1038/s41586-026-1… 5. AlphaGenome Atlas doi.org/10.64898/2026.09.16.… 6. Claude uplifting biomolecular modeling anthropic.com/research/claud… 7. Pinal doi.org/10.1038/s41586-026-1… 8. Virtual yeast doi.org/10.1038/s41586-026-1… 9. Digital organism doi.org/10.1038/s41591-026-0… 10. Generalist biological AI doi.org/10.1038/s41587-026-0…

i keep coming back to the systems work behind AI in biology Evo 2 uses StripedHyena 2 to model DNA at single-nucleotide resolution with a 1 million-token context window the architecture details are a pretty natural rabbit hole after optimizing GPU kernels

my favorite papers that i’ve read over the past month (biased) links below! 👇 1. Qwen-CUA: Native Computer Use for (almost) Everything - I like qwen, and I love computer use. no brainer. 2. FreeToken: Efficient Edge-Native MoE Serving with Bandwidth-Adaptive Execution - as an avid @ollama user, I had to check out the competition. This did. NOT disappoint. 3. Prime Agent: A Self-Improving RLM Harness - I love long horizon research, this is probably my favorite paper on it. 4. KernelFoundry: Hardware-aware evolutionary GPU kernel optimization - when I got into the open kernel challenge, this is the first paper I read when I hit a roadblock. 5. Human Learning about - wanted to check out @raphaelroux’s work, probably my favorite paper over the past 3 months last but not least… shoutout @askalphaxiv. I read all the papers through their mobile app on the way to school every day. all love!