You’re a soldier in the field, and you ask your personal AI assistant how to destroy a bridge or how much ketamine dosage to administer to an injured teammate. What if your assistant told you “sorry, I can’t help you with that. I can’t condone violence," or "I can’t assist with drug use.” In time critical, life or death situations, an AI assistant that refuses to answer the warfighter's query is useless, or worse than useless if it’s intended to be a trusted part of the kit. In this talk I discuss EdgeRunner's work on developing new benchmarks to measure LLMs' propensity for refusing military queries, as well as our work to mitigate these refusals, including our strategy to build military LLMs from the ground up. Blog post and paper: edgerunnerai.com/news/when-a…

PHLoRA: A Paper Review In this talk I discuss the PHLoRA technique, in which LoRA matrices can be extracted from fully-fine-tuned models to enable substantially more efficient inference when hosting fine tuned models from many customers. I participated in this project with my colleagues from Amazon while I was there. Nice work, Bhoomit, Anjie, and Sushmit. PHLoRA: arxiv.org/abs/2509.10971 S-LoRA: arxiv.org/abs/2311.03285

REFRAG is a new model from the Meta Superintelligence Lab, which uses a lightweight encoder to compress the retrieved context from a data store, thereby reducing the context length fed into the LLM. It also includes an RL policy to dynamically decide which chunks of context should be compressed through the encoder, versus which should be fed to the LLM unchanged. In this talk I discuss the paper, the results, and conclusions. Paper: arxiv.org/abs/2509.01092

Engineering a performant AI system is all about tradeoffs. As one example, when creating a vector store over which to perform retrieval augmented generation, what size of embeddings should you choose? Researchers at DeepMind sought to characterize the limitations of embedding based retrieval systems as a function of the embeddings size, providing both theoretical analyses and a new benchmark called LIMIT. In this talk we'll discuss the paper, the broader context, and applications. On the Theoretical Limitations of Embedding-Based Retrieval Paper: arxiv.org/abs/2508.21038 Repo: github.com/google-deepmind/l… Factual Knowledge Acquisition in Pretraining: arxiv.org/abs/2406.11813 Hybrid Search: techcommunity.microsoft.com/… Matryoshka Models: research.google/pubs/matryos…

Activated LoRAs, or aLoRAs, allow a foundation model system to switch between LoRA adapters partway through a multiturn interaction, including for agentic tool use. By eliminating the need to recompute the KV cache, ALoRAs can speed up the overall interaction by up to 35x when performing a mid-interaction adapter change, all while not losing task performance. aLoRA paper: arxiv.org/abs/2504.12397 activated-lora repo: github.com/IBM/activated-lor… a blog post on KV-caching: huggingface.co/blog/not-lain… Hat tip to our CTO Vincent for highlighting this work to me.

I’m pleased to announce that I’ve joined @EdgeRunnerAI as Chief Science Officer. Talent is the most precious resource in AI development, and I am thoroughly impressed by the caliber of builders and leaders I’ve met at Edgerunner AI. I am honored to be a part of this team. Our initial focus is to build resilient AI models that can run on any device and without network access. Our customers expect the same richness of capability as that of cloud-based frontier models while possessing only a tiny fraction of the compute and memory on their edge devices. Through innovation we will deliver. Some of our research work includes distillation, quantization, edge and federated learning, new RAG approaches, novel RL signals, new model architectures, and more, which is all on top of core model building activities like data acquisition and preparation, model training, and evaluation. We are frontier AI builders, and we relentlessly prioritize the specialized needs of our warfighters. Our immediate work on edge models harmonizes with our strategic ambitions, which I might call “transparent and self-sufficient American AI.” AI technology is increasingly fundamental to our national security, including hardware, software, and infrastructure. I am eager to partner with aligned companies and researchers, including everyone from modular nuclear power companies, to compute manufacturers, to other software builders, to device firms, toward building national AI self-sufficiency. Moreover, we at Edgerunner AI advocate model transparency, such that the training data and modeling techniques we use are auditable and understandable to our customers. America has always and must always lead the world in the free exchange of ideas, speech, and findings. My ancestors fought for this country, invested in it, and planted seeds that grew into the trees under which I now enjoy the shade. Before Amazon I was an Air Force officer, and I am excited to be of service to the American people once again. If you are currently a DoD member, an AI modeler, or anyone who shares our vision, I would love to speak with you. Let’s build a bright future for our posterity together.