Carlos Guestrin retweeted
I don't think people have realized how crazy the results are from this new TTT + RL paper from Stanford/Nvidia.
Training an open source model, they
- beat Deepmind AlphaEvolve, discovered new upper bound for Erdos's minimum overlap problem
- Developed new A100 GPU kernels 2x faster than the best human kernel
- Outperformed the best AI coding attempt and human attempt on AtCoder
The idea of Test Time Training is to train a model *while* it's iteratively trying to solve a task. Combining this with RL like they do in this paper opens up the floodgates of possibilities for continual learning
Authors: @mertyuksekgonul @LeoXinhaoLee @JedMcCaleb @xiaolonw @jankautz @YejinChoinka @james_y_zou @guestrin @sun_yu_
Carlos Guestrin retweeted
Excited to share our release of LOTUS 1.1.0, which now makes it easier than ever to get started with LLM-powered data processing over a variety of custom knowledge sources.
Seamlessly connect to data from the web, your SQL databases, vector databases and more.
And process it all with the power of semantic operators.
github.com/lotus-data/lotus
Carlos Guestrin retweeted
SAIL is delighted to announce Carlos @guestrin, the Fortinet Founders Professor of Computer Science, as the next Director of @StanfordAILab. Carlos is a talented researcher and leader, known for his work on explainability, graphs, compilation, and boosted trees in AI.
Carlos Guestrin retweeted
We've been building LOTUS at Stanford and Berkeley to make LLM-powered data processing fast, easy and declarative.
LOTUS is an open-source query engine that makes programming as easy as writing Pandas and optimizes your programs for up to 400x speedups.
To celebrate the holidays, we're excited to share our release of LOTUS 1.0.0 with a batch of new updates that make reasoning over your data faster, easier and better than ever!
Code: github.com/guestrin-lab/lotu…
🧵👇
Carlos Guestrin retweeted
Can interpretability help defend LLMs? We find we can reshape activations while preserving a model’s behavior. This lets us attack latent-space defenses, from SAEs and probes to Circuit Breakers. We can attack so precisely that we make a harmfulness probe output this QR code. 🧵
Carlos Guestrin retweeted
🧵LLMs are great at synthesizing info, but unreliable at citing sources. Search engines are the opposite. What lies between them?
Our new paper runs human evals on 7 systems across the✨extractive-abstractive spectrum✨for utility, citation quality, time-to-verify, & fluency!
Carlos Guestrin retweeted
Super excited to share our work on LOTUS, a query engine for reasoning over large corpuses of data with LLMs!
Joint work w/ the amazing @sid_jha1, @matei_zaharia & @guestrin
Read the paper: arxiv.org/abs/2407.11418
Try out the code: github.com/stanford-futureda…
🧵👇
Carlos Guestrin retweeted
Prior work has used LLMs to simulate survey responses, yet their ability to match the distribution of views remains uncertain.
Our new paper [arxiv.org/pdf/2411.05403] introduces a benchmark to evaluate how distributionally aligned LLMs are with human opinions.
🧵
Carlos Guestrin retweeted
Many providers offer inference APIs for the same models: for example, there were over nine Llama-3 8B APIs in Summer 2024. Do all of these APIs serve the same completion distribution as the original model?
In our new paper, ✨Model Equality Testing: Which Model is This API Serving?✨, we formalize this question as a two-sample distribution testing problem: the user collects samples for their task from the API and a reference distribution, and conducts a statistical test to see if the two distributions are the same.
We design tests which show nontrivial power in detecting when models have been quantized, watermarked, or finetuned! (🧵 1/5)
Carlos Guestrin retweeted
It’s happening! TVM Conference. Incredibly proud of and grateful to the whole UW SAMPL team and TVM contributors.
Carlos Guestrin retweeted
"if... users do not trust a model or a prediction, they will not use it" -@sameer_ @guestrin arxiv.org/abs/1602.04938v1
Carlos Guestrin retweeted
LIME, an algorithm that can explain the predictions
of any classifier or regressor in a faithful way by @guestrin goo.gl/OdyG1i
Carlos Guestrin retweeted
Dato GraphLab demo yesterday CSIRMDS. Looking forward to trying it out cc @datoinc @nunuska @vasiloglou #datascience
Carlos Guestrin retweeted
7 Python Tools All Data Scientists Should Know How to Use ow.ly/UTnBO


























