Ashutosh Mehra retweeted
How can we train LLMs to write memories from experience collected on the fly to as best as possible solve the next task, i.e., optimize for both capability and "continual learning" ability?
In our new post, we study this question in the setting continual learning over a sequence of tasks, and develop a simple approach to train models to write effective memories
cmu-aire.github.io/RIME/
*Spoiler*: the picture is more nuanced than you think 🧵 ⬇️
Ashutosh Mehra retweeted
New blog!
This one covers my experiments on fine-grained credit assignment with an agentic judge, and how it affects the models' performance and reasoning quality and some thoughts on RL science.
Link below, please do check it out, and RT/QT if you like it:)
Ashutosh Mehra retweeted
When Audience Asked Richard Feynman,
Do you think there will ever be a machine that will think like human beings and be more intelligent than human beings?
Ashutosh Mehra retweeted
RL has reached another local optimum. It keeps climbing.
interconnects.ai/p/reverse-e…
Ashutosh Mehra retweeted
I've been collecting all sorts of extra information about the new OpenAI structured outputs feature throughout the day
I think my post here may now be the definitive guide to details you may have missed
simonwillison.net/2024/Aug/6…
Some cool/crazy lessons:
- don't mention pink elephants if you don't want LLMs to think about them
- the one by @charles_irl of actually using bad data (labeled as such) is 🤯
Also: This is a nice, low-friction format for sharing one-off lessons!
✨ Today I went to one of the highest signal events in SF.
AI engineers came together to share their unique hacks/lessons learned on how to build with LLMs. If you are a dev, listen up!
At “So you think you can prompt” event hosted by @BEBischof and @slaterstich at @BainCapVC’s new SF office.
🧵 👇
1
10
2,448
Interesting visualizations showing LLMs representing:
- categorical concepts as simplices
- hierarchical relations as orthogonality
Fundamentally, high-level concepts group into categorical variables---mammal, reptile, fish, bird---with a semantic hierarchy---poodle is a dog is a mammal is an animal.
How do LLMs internally represent this structure?
arxiv.org/abs/2406.01506
7
1,233
Ashutosh Mehra retweeted
I gave a lecture at @Stanford CS 25.
Lecture video: youtu.be/orDKvo8h71o?si=RIfy…
AI is moving so fast that it's hard to keep up. Instead of spending all our energy catching up with the latest development, we should study the change itself.
First step is to identify and understand the dominant driving force behind the change. For AI, a single driving force stands out; exponentially cheaper compute and scaling of progressively more end-to-end models to leverage that compute.
However this doesn’t mean we should blindly adopt the most end-to-end approach because such an approach is simply infeasible. Instead we should find an “optimal” structure to add given the current level of 1) compute, 2) data, 3) learning objectives, 4) architectures. In other words, what is the most end-to-end structure that just started to show signs of life? These are more scalable and eventually outperform those with more structures when scaled up.
Later on, when one or more of those 4 factors improve (e.g. we got more compute or found a more scalable architecture), then we should revisit the structures we added and remove those that hinder further scaling. Repeat this over and over.
As a community we love adding structures but a lot less for removing them. We need to do more cleanup.
In this lecture, I use the early history of Transformer architecture as a running example of what structures made sense to be added in the past, and why they are less relevant now.
I find comparing encoder-decoder and decoder-only architectures highly informative. For example, encoder-decoder has a structure where input and output are handled by separate parameters whereas decoder-only uses the shared parameters for both. Having separate parameters was natural when Transformer was first introduced with translation as the main evaluation task; input is in one language and output is in another.
Modern language models used in multiturn chat interfaces make this assumption awkward. Output in the current turn becomes the input of the next turn. Why treat them separately?
Going through examples like this, my hope is that you will be able to view seemingly overwhelming AI advances in a unified perspective, and from that be able to see where the field is heading. If more of us develop such a unified perspective, we can better leverage the incredible exponential driving force!
Slides: docs.google.com/presentation…
Ashutosh Mehra retweeted
Heard about LM program optimization in DSPy but not sure how it works or where to start?
Check out our new paper on *how* to optimize LM program instructions and demos!
Packaged with our new SOTA LM program optimizer MIPROv2! Threads and notebooks coming later this week!
🚨Announcing the largest study focused on *how* to optimize the prompts within LM programs, a key DSPy challenge.
Should we use LMs to… Craft instructions? Self-generate examples? Handle credit assignment? Specify a Bayesian model?
By @kristahopsalong* @michaelryan207* &team🧵
Ashutosh Mehra retweeted
This was a fun paper
Replying to @ericzelikman
Hypothesis Search strengthens LLMs’ inductive reasoning:
1) Given training pairs, prompt LM to come up with hypotheses for the underlying rule
2) For each hypothesis, prompt LM to implement it in code
3) Run the code on training pairs. Revise if it errors; else submit for testing
These pop-out bezel animations in iOS 18 beta... could they be setting the stage for the upcoming iPhone 16 having solid state buttons? 🤔
These animations + haptics would then compensate for the feel of pressing physical buttons.
1
6
1,194
Ashutosh Mehra retweeted
New blog: Can LLM-based eval replace human evaluation?
ehudreiter.com/2024/06/11/ll…
Ashutosh Mehra retweeted
Yesterday Apple strongly advocated for on-device AI with @Apple Intelligence! Today, we know they will run ~3B LLM on device (Mac, iPhone, iPad) using fine-tuned LoRA Adapters for different tasks, claiming to outperform other 7B and 3B LLMs! 👀
What we know:
📱 On-device 3B with grouped-query-attention, activation and embedding quantization running on the neural engine
📊 iPhone 15 Pro 0.6 ms time-to-first-token with 30 tokens/second latency
❌ Server model size and details are unknown
🔄 Dynamically load, cache, and swap LoRA adapter models as needed
📈 On-device model 49K vocab size, while the server model 100K
🛠️ Used Rejection Sampling and descent policy optimization and a leave-one-out advantage for RLHF
📝 Used synthetic data generation for tasks like summaries
🔍 750 evaluation samples for each production use case
🇺🇸 Seems to be an English model, this would explain the limitation to US only
🧠 Used Apple's AXLearn framework (JAX) and FSP to train on TPUs and GPUs
⚡ 3B + Adapter outperforms Phi-3 mini on summarization
🏆 3B + Adapter achieves 78.7% on IFEval > Mistral 7B; Server Model matches GPT-4-Turbo
Apple has made several improvements for on-device ML in iOS 18.
- model compression improvements in palettization, quantization, pruning, and post-training compression
- transformers support improvements via in-place state tensors for KV Cache, SDPA ops, multiple adapters
🧵
Session link: developer.apple.com/videos/p…
218


























