ML research @ google deepmind Zürich vision things in gemini opinions are mine but registered copyright of my mother
Zürich, Schweiz
Joined November 2019
- Tweets208
- Following128
- Followers1.7K
- Likes331
Basil Mustafa retweeted
Gemma 3 is best in class for a VLM that runs on 1 GPU. Should make RL fine tuning feasible. Also Academic researchers can apply for Google Cloud credits (worth $10,000 per award) to accelerate their Gemma 3-based research.
Introducing Gemma 3: our open model that runs on single GPU/TPU! SoTA AI for its size; multimodal; supporting 140 languages; long context of 128k; fast and high-performance with quantization; open weights; function calling. What more do you want? @google blog.google/technology/devel…
Basil Mustafa retweeted
Dang, Google's veo 2, same prompt.
Basil Mustafa retweeted
Have you ever wondered how to train an autoregressive generative transformer on text and raw pixels, without a pretrained visual tokenizer (e.g. VQ-VAE)?
We have been pondering this during summer and developed a new model: JetFormer 🌊🤖
arxiv.org/abs/2411.19722
A thread 👇
1/
Basil Mustafa retweeted
Gemini 2.0 Flash ⚡️ has arrived!
2.0 Flash > 1.5 Pro (again!) 📈
Interacts with a browser 🤖
Native image generation 🖼️
and much more!
Try it out aistudio.google.com/prompts/…
As a preview of what is possible, wishing you all a Drastic Holiday powered by 2.0!
Basil Mustafa retweeted
I was so impressed with the Astra demo at Google I/O yesterday that I decided to build my own version using Gemini 1.5 Pro Flash.
It's so fast and really good. ⚡️
It was even able to detect the gate! Content is streamed directly from my camera.
Voice via @elevenlabs
Basil Mustafa retweeted
This is something I've worked on for a while! You can save the activations of one LLM call and reuse them for a follow-up that overlaps with the first.
This means asking a question about a big codebase can take 30 seconds the first time and 1s after that!
Gemini’s context caching is one of the most exciting releases that came out it of Google I/O. ai.google.dev/gemini-api/doc…
Basil Mustafa retweeted
Mistral and Phi are juicing to get higher benchmark numbers, while GPT, Claude, Gemini, and Llama are not.
arxiv.org/abs/2405.00332
Basil Mustafa retweeted
Bunch of improvements to Gemini 1.5 Pro today:
* No waitlist on the API.
* Better at reasoning, multimodal, instruction-following and a bunch of other evals.
* Video+Audio support: Video inputs can now parse audio as well on AIStudio.
developers.googleblog.com/20…
Basil Mustafa retweeted
Following our previous work, we are releasing RecurrentGemma - a fully open source 2B model based on our Griffin architecutre!
Code + weights as everyone has wished for!
Code on Github: github.com/google-deepmind/r…
Weights on Kaggle: kaggle.com/models/google/rec…
Basil Mustafa retweeted
I’m proud to announce our launch of @bioptimus_ai – with a mission to create and build the first #foundationmodel for biology.
(1/3)
Basil Mustafa retweeted
What the fuck...
On a benchmark measuring precision recall from a FULL DAY OF AUDIO, Gemini Pro 1.5, when just seeing the *audio file* directly (no transcription!) far outperforms GPT-4 with a Whisper transcription.
This is mind-bending.
Basil Mustafa retweeted
Finally got to test SigLIP models and loss. The model is SO GOOD! 10% performance increase with a smaller visual encoder!
The loss also works - SigLIP is slightly better than standard CLIP loss (on screenshot).
Can't wait to finish training new NLLB-CLIP models!
Basil Mustafa retweeted
very underrated @Gradio feature is Annotated Image component, see it in action in OWLv2 👉 huggingface.co/spaces/merve/…
Basil Mustafa retweeted
Bard has had such capabilities for months now, yet people are more excited about GPT-4V.
Shows how soft power and narrative is more important than technology itself.
Makes you think whether the interest in such advances is genuine or shiny toy syndrome.
g.co/bard/share/847bd55263b3
Basil Mustafa retweeted
Next time you train a contrastive model in open_clip, give SigLIP a try: you may be able to train a larger and/or cheaper, slightly better model!
Or SigLiT to train a text model for a given pre-trained "semantic-free" image model like DINO or MAE.
arxiv.org/abs/2303.15343
SigLIP PR is here github.com/mlfoundations/ope… ... any wtfs there @giffmana ? :) we don't have jax.lax.ppermute in PyTorch land, so had to roll my own, sniff. Adding a custom autograd.Fn impl of the exchange seemed to get things working...
Basil Mustafa retweeted
For those interested in a high-level overview, a quick summary of our work on the Pathways Language Image model on Google ResearchBytes.
youtube.com/watch?v=K12bYQzU…
Basil Mustafa retweeted
Sparse MoEs give strong predictive perf., decoupled from compute! But how robust & calibrated are they? Do they know what they don't know? Can we do better?
We tackle these Qs in the context of CV in our TMLR paper "Sparse MoEs meet Efficient Ensembles" openreview.net/forum?id=i0ZM…
Sparse MoEs meet Efficient Ensembles
James Urquhart Allingham, Florian Wenzel, Zelda E Mariet et al.
openreview.net/forum?id=i0ZM…
Basil Mustafa retweeted
With the growing adoption of #AI, mitigating unintended bias is essential. Do fairness properties transfer? How do we debias with arbitrary # of classes/groups? Can debiasing be made interpretable? Below is a short thread about 3 ML fairness papers to appear @ #NeurIPS2022 [1/5]