Samira Abnar (@samira_abnar) profile picture

Apple ML research

San Francisco, CA
Joined June 2024
Photo by Samira Abnar (@samira_abnar)Photo by Samira Abnar (@samira_abnar)Photo by Samira Abnar (@samira_abnar)Photo by Samira Abnar (@samira_abnar)Photo by Samira Abnar (@samira_abnar)Photo by Samira Abnar (@samira_abnar)Photo by Samira Abnar (@samira_abnar)Photo by Samira Abnar (@samira_abnar)
🚨 One question that has always intrigued me is the role of different ways to increase a model's capacity: parameters, parallelizable compute, or sequential compute? We explored this through the lens of MoEs:
4
65
8
282
48,565
Samira Abnar retweeted
We’ve raised a $1B Series A, led by @kleinerperkins and @clearlake. Capital is a means, not an end. Our goal is radical leverage for scientists: One person doing what took a team. A team doing what took an institution.
Today we're announcing our $1B Series A, led by @kleinerperkins and @clearlake. We believe making frontier AI R&D more accessible is the most direct path to AI that solves humanity's most pressing problems across science and technology.
92
45
14
924
99,799
Samira Abnar retweeted
Replying to @pushideh
تمام کتابهای یوال حراری. حرف‌های غلط و درست و قاطی ولی مضر به بیان جذاب و شیوا! در این حد که سه تا کتابشو باهم خریدم بخونم نصف یکی رو خوندم کتابهارو انداختم تو ریسایکل! همینطور مالکوم گلدول، سم هریس و دلقک های دیگه‌ی اکیپشون.
1
1
16
474
Samira Abnar retweeted
خاک بر سر بیسوادتون کنن، این حرکت تیم ملی کنگو بر علیه متجاوز خارجیه نه ظلم داخلی و به ایستادن تیم ملی با کوله‌پشتی خیلی شبیه‌تره! اون کسایی که مردم کنگو رو میکشن گروه‌های تروریستی ADF و M23 و غیره ته دمشون میرسه به امارات و آمریکا و چین از طریق رواندا و اوگاندا و برونئی، و همه‌ش ناشی از اسپانسرهای خارجیه که دندون برای معادن طلا و لیتیوم کنگو برای تکنولوژی تیز کرده‌ند.
در اوایل سال ۲۰۲۴ بازیکنان تیم ملی‌ کنگو در یک اکت اعتراضی و زیبا موقع سرود ملی کشورشون برای حمایت از مردم و جلب توجه افکار جامعه جهانی به کشتار هموطنانشون این حرکت رو انجام دادن.
1
6
1
45
1,388
Samira Abnar retweeted
به عنوان یک زن ایرانی کنار زنان افغانستان می‌ایستم، همان‌طور که آن‌ها کنار ما ایستادند و می‌ایستند. زن، زندگی، آزادی #هرات
30
41
7
582
16,122
Samira Abnar retweeted
Want to send Trump a message before he escalates the war with Iran? Buy gas today. High oil prices are his kryptonite. Do it now. 🛢️ #PumpUpThePressure #NoWarCrimeslnIran
9
1
23
704
Samira Abnar retweeted
Just speechless. how would Americans feel if Iran bombed mayo clinic?! #StopWarOnIran
The aggression against Pasteur Institute of Iran—a century-old pillar of global health & member of International Pasteur Network—is a direct assault on international health security. This violates Geneva Conventions & IHL principles. We call on @WHO @ICRC & global health bodies to condemn this attack, assess damages & support reconstruction. #PasteurInstituteIran #HealthSecurity #GenevaConventions #InternationalLaw #GlobalHealth #WHO #ICRC #HumanitarianLaw #Iran #ReconstructHealth #PasteurNetwork
16
52
1,104
با دقت توجه کنید مطمئن باشید که جاسوس‌های اسرائیل در ایران تحصیل‌کردهٔ آمریکا نیستند. آن‌ها تظاهر می‌کنند انقلابی هستند و حتی در زمان جنگ هم با دامن زدن به فتنه، خودشان را لو می‌دهند.
This Post is from a suspended account. Learn more
42
34
2
307
27,546
Samira Abnar retweeted
اشک غیرت دکتر رحیمی مدیر پروژه بزرگترین پل خاورمیانه در کرج: این پروژه پروژه ای بود برای مردم و عمومی بود نه استفاده نظامی میشد ونه هیچ یک از دروغ هایی که رسانه ها گفتند مصداق دارد. این کارگاه مهندسی است. من ناراحت این هستم که بد قول در برابر مردم شدم. ولی این آرزو را به گور خواهند برد که این پل بلا استفاده شود.
1
155
2
1,031
57,344
Samira Abnar retweeted
آخرش من با این گریه م گرفت. آقا بزارین ما بیایم ایران با خیال راحت کار و زندگیمون رو بکنیم. تخصصمون رو بزنیم به زخم مملکت خودمون. ما نه وطن فروشیم نه خائن، هر مخالفتی هم با شما داریم به عشق ایرانه. چند نفر از دوستانم رو باید ببینیم که برگشتن تا برای ایران کار کنند و به اتهام جاسوسی کار و خانه و زندگیشون رو به باد دادید و دوباره آواره کردید؟ یکی برای آوردن واکسن HPV، یکی برای آوردن استاد بین الملل تئاتر، یکی برای چاپ کتاب ... این چه سرنوشتیه؟ متنفرم از زندگی در کشوری که داره پدر مادرم رو بمباران میکنه!‌ میفهمید یعنی چی؟ #نه_به_جنگ
با دقت توجه کنید مطمئن باشید که جاسوس‌های اسرائیل در ایران تحصیل‌کردهٔ آمریکا نیستند. آن‌ها تظاهر می‌کنند انقلابی هستند و حتی در زمان جنگ هم با دامن زدن به فتنه، خودشان را لو می‌دهند.
3
17
1
139
10,123
Samira Abnar retweeted
سازمان منابع طبیعی و آبخیزداری استان تهران که در بمباران امروز رسماً پودر شد. این‌جا هم داشتن موشک و بمب اتم می‌ساختن هموطن؟ چرا هرجای دنیا که درباره ویرانی‌های ایران خبر کار شده یک ایرانی نوشته این‌جا پاسدارها مخفی شدن/حتما این‌جا موشک می‌ساختن؟ بس کنید این بساط وقاحت و پستی و رذالت رو
125
225
14
966
49,513
Samira Abnar retweeted
For years I watched children of Gaza suffer. Helplessly all my poor heart could do was to study their faces, and every word, promising them I'd remember them forever. And that's how I know what Israel is planning for my home country and our children now.
“They killed my mother — and before that, my brother. I urge everyone: do not go to the American aid distribution points soaked in blood. This bag only contains 2 empty flour sacks — nothing else — & yet they shot & killed my mother as she tried to get us something to eat today.”
4
14
42
2,332
So I understand that was unexpected for a lot of people, @Apple MLR has released a protein folding model! arxiv.org/abs/2509.18480. Here’s a summary of what SimpleFold is and what it represents: - What is SimpleFold? A generative model that essentially treats protein folding almost exactly as if it were a text-to-image or text-to-3D problem. - What are we sharing? A research paper and a codebase under an MIT license github.com/apple/ml-simplefo… (looking forward to people contributing to it!). We are also releasing pre-trained checkpoints of different sizes so that researchers can best tradeoff performance for efficiency. - Why protein folding? We are doing this work largely because protein folding is an excellent benchmark for structured data generation and multi-modality. Protein folding is a very interesting problem from a generative modeling perspective and we do research on generative modeling :) - Why is it interesting? IMO SimpleFold is interesting because I believe in finding recipes (architectures, training objectives, etc.) that generalize across the board to many different data modalities. Let’s say you are an ML expert in text-to-image or text-to-3D, now you can apply your latest and greatest architectural blocks or efficient samplers to protein folding with SimpleFold. I believe this is a net benefit for ML research and science in general. Now getting more into the technical details: - Our architecture is very simple (hence the name), just a stack of transformer blocks with time-step conditioning. This is important because it makes the model efficient at inference time. You can run SimpleFold directly on your Mac and get results quickly without data ever leaving your laptop. - SimpleFold is not necessarily a model that “rejects” inductive biases, it just doesn’t enforce them directly on the architecture. For example, we apply rotation augmentation to all the protein structures during training. This makes the model “softly” invariant to this symmetry. - There were some concerns online about data leakage from AFESM and that driving performance of SimpleFold or making it overfit. We filtered AFESM data so that the CASP14 sequences are not seen during training. As a matter of fact we distilled structures from AF2/ESMFold models, which have the same cutoff data as SimpleFold for PDB data. Both AF2 and ESMFold train on self-distilled datasets, we just train SimpleFold on a bigger set of distilled data. I want to thank my awesome team of collaborators, they are all rockstars. That’s all, for now :)
SimpleFold: Folding Proteins is Simpler than You Think "we introduce SimpleFold, the first flow-matching based protein folding model that solely uses general purpose transformer blocks. Protein folding models typically employ computationally expensive modules involving triangular updates, explicit pair representations or multiple training objectives curated for this specific domain. Instead, SimpleFold employs standard transformer blocks with adaptive layers and is trained via a generative flow-matching objective with an additional structural term."
7
40
3
309
50,744
Last year at @Apple MLR, we published a number of interesting papers like AIM, AIMv2, and Scaling laws for: Sparsity, Native Multimodal Models, Data mixing. Today the team has open-sourced the training codebase we used for conducting this research! github.com/apple/ml-l3m
4
55
3
437
28,634
We propose new scaling laws that predict the optimal data mixture, for pretraining LLMs, native multimodal models and large vision encoders ! Only running small-scale experiments is needed, and we can then extrapolate to large-scale ones. These laws allow 1/n 🧵
6
48
7
263
31,476
Samira Abnar retweeted
We have two awesome new videos on MLX at #WWDC25 this year. - Learn all about MLX. - Learn all about running LLMs locally with MLX. @angeloskath, @shshnkp, myself, and others worked super hard to make these. Check them out and hope you find them useful!
25
52
6
430
90,471
Check out this post that has information about research from Apple that will be presented at ICLR 2025 in 🇸🇬 this week. I will be at ICLR and will be presenting some of our work (led by @samira_abnar) at SLLM @sparseLLMs workshop. Happy to chat about JEPAs as well!
5
18
2,516
Samira Abnar retweeted
Our work on fine-grained control of LLMs and diffusion models via Activation Transport will be presented @iclr_conf as spotlight✨Check out our new blog post machinelearning.apple.com/re…
Thrilled to share the latest work from our team at @Apple where we achieve interpretable and fine-grained control of LLMs and Diffusion models via Activation Transport 🔥 📄 arxiv.org/abs/2410.23054 🛠️ github.com/apple/ml-act 1/9 🧵
1
10
1
40
3,012
We release a large scale study to answer the following: - Is late fusion inherently better than early fusion for multimodal models? - How do native multimodal models scale compared to LLMs. - How sparsity (MoEs) can play a detrimental role in handling heterogeneous modalities? 🧵
10
83
14
460
87,149
Training and scaling large multimodal models from scratch? This is the thread for you. In this new paper, we provide an extensive study with hundreds of runs, fitting scaling laws for early/late fusion models, MoEs, and exploring different data mixtures. Tons of cool findings.
We release a large scale study to answer the following: - Is late fusion inherently better than early fusion for multimodal models? - How do native multimodal models scale compared to LLMs. - How sparsity (MoEs) can play a detrimental role in handling heterogeneous modalities? 🧵
3
11
1
90
7,490
Samira Abnar retweeted
We’re looking for an intern at Apple MLR 🍎 ASAP. Join us if interested in building universal diffusion/flow-matching model at scale!
We are looking for an intern to join MLR at Apple ASAP. You will be working with me, @YuyangW95 and other scientists in the team on domain-agnostic generative modeling at scale. If you are interested please reach out to mbautistamartin at apple by the end of this week.
1
9
68
15,359