University of Calgary Machine Learning Lab
Calgary, AB
Joined June 2021
- Tweets18
- Following46
- Followers58
- Likes44
Calgary ML Lab retweeted
📄 Paper: arxiv.org/abs/2611.10866
🌐 Project page: agenticlearning.ai/deleting-…
💻 Code: github.com/agentic-learning-…
🤗 Trained masks: huggingface.co/Jacklu0831/de…
So grateful to have co-led this with @yangzb159 and worked with @mikelasby, @Tejas_915, @yanii, and @mengyer at NYU's @agentic_ai_lab and @UCalgaryML!
This was a fun discovery, and it still surprises us that something as simple as deleting weights just worked on everything we tried, across model sizes and RL, SFT, and SDFT. See you at NeurIPS!
Calgary ML Lab retweeted
New from our lab, led by @Jacklu_me and @yangzb159 with @yanii’s group at @UCalgaryML: zeroing <0.1% of an LLM’s weights matches full fine-tuning at scale and avoids forgetting!
The Surprising Effectiveness of Deleting Weights in LLM Post-Training ✂️: arxiv.org/abs/2611.10866
Even the simplest weight edit, a deletion, can teach an LLM new skills!
Meet Deletion Tuning (#NeurIPS2026): just zeroing <0.1% of weights matches full fine-tuning's accuracy at scale. It's thousands of times smaller and avoids forgetting, a step toward continual learning. 🧵👇
Calgary ML Lab retweeted
Olá, ICLR 2026 in Rio de Janeiro! 🇧🇷
Excited to present our paper, “REAP the Experts: Why Pruning Prevails for One-Shot MoE compression” this Friday from 10:30-13:00 at Pavilion 3 (P3-#701).
REAP is a one-shot MoE pruning algorithm that scales to 1T params. Details 🧵👇
Calgary ML Lab retweeted
No tricks, just treats from our ML team. 👻
(1/5) Compressing a 1T parameter MoE model by 50% with virtually no loss in quality is now possible.
Our new paper with @mikelasby @lzrvch @NishSinnadurai Sean Lie and @yanii introduces an expert pruning criterion for one-shot compression that aims to solve the memory bottleneck of these massive models, making them far more accessible for research and deployment.
Calgary ML Lab retweeted
@UCalgaryML will be at #ICML2025 in Vancouver🇨🇦 next week: our lab has 6 different works being presented by 5 students across both the main conference & 6 workshops!
We are also recruiting fully-funded ML PhD/MSc students🧑🎓If attending, reach out to me on Whova app to meet 📅 – at Vancouver, British Columbia
ALT The Calgary Machine Learning Lab, will have 6 different works being presented by 5 students across the International Conference on Machine Learning (ICML) 2025 workshops (HiLD, ES-FoMo III, DIG-BUGS, WiML, MusiML) and main conference! Please see our full conference schedule on the lab website for more details.
Calgary ML Lab retweeted
1/10 🧵
🔍Can weight symmetry provide insights into sparse training and the Lottery Ticket Hypothesis?
🧐We dive deep into this question in our latest paper, "Sparse Training from Random Initialization: Aligning Lottery Ticket Masks using Weight Symmetry", accepted at #ICML2025
Calgary ML Lab retweeted
I'm proud that the @UCalgaryML lab will have 6 different works being presented by 6 students across #NeurIPS2024, in workshops (@unireps, @WiMLworkshop, MusiML) and the main conference! 🎉
Hope to see you at our posters/talks 🤓, full schedule at calgaryml.com
🧵(1/4) – at Calgary, Alberta
Calgary ML Lab retweeted
For more details, check out our full paper … 👇🏼
🔗: neurips.cc/virtual/2024/1026…
Joint work w/ @adnan_ahmad1306, @EkanshSh, and @yanii from @VectorInst and @UCalgary.
Visit our poster session on Dec 14 at #NeurIPS2024! 👋🏻
Supported by: @UCalgaryML
(6/6)
Calgary ML Lab retweeted
Really excited to be presenting this work from my lab at the @NeurIPSConf 2024 @unireps workshop! 🎉
If you are at NeurIPS and interested in the Lottery Ticket Hypothesis/sparse training, don’t miss our poster on Sat Dec 14th, 3:45-5:00 PM 🖼️🤓
@UCalgaryML @VectorInst
✨Our new @unireps paper tries to answer why the Lottery Ticket Hypothesis (LTH) fails to work for different random inits through the lens of weight-space symmetry.
We improve the transferability of LTH masks to new random inits leveraging weight symmetries.
🧵(1/6)
Calgary ML Lab retweeted
Congratulations @adnan_ahmad1306 on receiving the @RBCBorealis Fellowship!
Adnan is a PhD student in the @UCalgaryML lab at @SchulichENGG/@UCalgary, and @VectorInst, co-supervised by @rahulgk.
🌟🎓 Meet @RBCBorealis's 2023-2024 Fellows! We're excited to introduce 10 exceptional #PhD students selected for our #Fellowships program, representing the brightest talent from Canada's top universities.
Learn more about the Fellows and their #research: rbcborealis.com/news/celebra…
Calgary ML Lab retweeted
Happy to share our work on Knowledge Distillation temperature🌡️ and fairness⚖️. Must-read for AI practitioners given the prevalence of distilled models! KD is crucial in deploying AI, e.g. limited hardware, real-time applications
🙏@sarahookr @mikelasby for feedback! @UCalgaryML
Knowledge #distillation is a widely used model compression method. We explore the nuanced impact of temperature on distilled models' #fairness ⚖️. Our findings show distilled students are less fair than their teachers at typical temps, but can be more fair in some instances…🧵👇
Calgary ML Lab retweeted
"Dynamic Sparse Training with Structured Sparsity" (openreview.net/forum?id=kOBk…) was accepted at ICLR 2024! DST methods learn state-of-the-art sparse masks, but accelerating DNNs with unstructured masks is difficult. SRigL learns structured masks, improving real-world CPU/GPU timings
ALT Real-world GPU batched inference timings for a linear layer extracted from ViT-B-16 trained with SRigL with a batch size of 256.
Calgary ML Lab retweeted
I’m excited to share that I’ve received an Amazon Research Award for my proposal "Addressing Catastrophic Forgetting with Dynamic Sparse Training" at @UCalgaryML/@UCalgary!
Learn more about the program on the @AmazonScience website: amzn.to/ara-fall-winter-2021 #AmazonResearchAwards
Calgary ML Lab retweeted
I was happy to recently be informed my 2022 DG proposal was also awarded a @NSERC_CRSNG and @NationalDefence DND Discovery Grant Supplement — one of only 20 awarded nationally — to further support and accelerate research at @UCalgaryML!
#DiscoveryGrants #NSERCsupport #CdnSci
Calgary ML Lab retweeted
Key takeaways:
- Initialization is important for older NNs: use sparsity aware init.
- For modern NNs, init doesn't matter: improving gradient flow during training helps (i.e. RigL)
- Lottery tickets work since they re-locate the pruning solutions.
Interested in understanding sparse training better? Join our oral presentation at #AAAI2022
"Gradient Flow in Sparse Neural Networks and How Lottery Tickets Win 🎟️" tomorrow (Feb 24) @ 10:30-11:45am PST! Poster at 8:45-10:30am.
Work with @yanii, @ynd, @cem_keskin_.
Calgary ML Lab retweeted
Work recently published by our close collaborators Drs. @marianapbento and @richardfrayneca!
Deep Learning in Large and Multi-Site Structural Brain MR Imaging Datasets frontiersin.org/article/10.3…
Calgary ML Lab retweeted
Join us for our #AAAI2022 *Oral Presentation* of our paper on Sparse DNN training: "Gradient Flow in Sparse Neural Networks and How Lottery Tickets Win" tomorrow (Feb 24) @ 10:30-11:45am PST!
Poster at 8:45-10:30am/Feb 27 4:45-6:30pm. Work with @utkuevci, @ynd, @cem_keskin_.
ALT AAAI Poster: Gradient Flow in Sparse Neural Networks and How Lottery Tickets Win