Research Scientist @apple AIML, Previously Postdoc @uwnlp & PhD @sbucompsc. Efficient #NLProc, on-device AI, mobile computing.
Seattle
Joined July 2014
- Tweets148
- Following617
- Followers582
- Likes465
Qingqing Cao retweeted
In this report we describe the 2025 Apple Foundation Models ("AFM"). We also introduce the new Foundation Models framework, which gives app developers direct access to the on-device AFM model.
machinelearning.apple.com/re…
Qingqing Cao retweeted
1/ KV Prediction for Improved Time to First Token
Tired of waiting forever for your on-device LLM to begin outputting tokens? I know I am.
In our latest preprint, we investigate a method for improving time to first token (TTFT) for on-device models by predicting a model’s KV cache using a small Auxiliary model. We demonstrate improvements in TTFT of up to 2x at a fixed accuracy. Full details: arxiv.org/pdf/2410.08391 . Code at github.com/apple/corenet/tre… .
Work done with @awk_ai , Chenfan Sun, Yanzi Jin, @sacmehtauw @morastegari Moin Nabi.
#LLM #Apple #Research #TTFT #Ondevice #OpenELM #Corenet #KVPrediction
Qingqing Cao retweeted
@awk_ai @HannaHajishirzi
Can’t run billion-level LLMs efficiently? Take a look at our work: APT.
We are excited to share our #ICML2024 oral paper, “APT: Adaptive Pruning and Tuning Pretrained Language Models for Efficient Training and Inference”.
Paper: shorturl.at/xabJl
Qingqing Cao retweeted
Like OpenELM, CatLIP is also "Open"
github.com/apple/corenet
Qingqing Cao retweeted
I'll be moderating an Efficiency in #NLProc Panel at the #SustaiNLP 🌿 workshop at #ACL2023 this Thursday. We have top leaders from Academia and Industry: @DrorRotem, @myleott, @sysnlp, @sarahookr
Any questions you would like to ask? Post here or join the panel!
Qingqing Cao retweeted
Don't miss our #ACL2023 tutorial on Retrieval-based LMs and Applications this Sunday!
acl2023-retrieval-lm.github.…
with @sewon__min, @ZexuanZhong, @danqi_chen
We'll cover everything from architecture design and training to exploring applications and tackling open challenges! [1/2]
ALT Tutorial on Retrieval-based LMs and Applications at ACL 2023. - Instructors: Akari Asai (University of Washington), Sewon Min (University of Washington), Zexuan Zhong (Princeton), Danqi Chen (Princeton) - Time & location: Sunday, July 9 14:00 - 17:30 (EDT) @ Metropolitan West - website https://acl2023-retrieval-lm.github.io/
Qingqing Cao retweeted
Introducing💃AdANNS: A Framework for Adaptive Semantic Search🕺
TL;DR: Up to 90× faster nearest neighbor retrieval and 2× lower memory cost for web-scale search.
Applies to vector search at scale & improves all "retrieval" augmented models!
arxiv.org/abs/2305.19435
[1/8]
Qingqing Cao retweeted
4-bit QLoRA is here to equalize the playing field for LLM exploration. You can now fine-tune a state-of-the-art 65B chatbot on one GPU in 24h.
Paper: arxiv.org/abs/2305.14314
Code and Demo: github.com/artidoro/qlora
QLoRA: 4-bit finetuning of LLMs is here! With it comes Guanaco, a chatbot on a single GPU, achieving 99% ChatGPT performance on the Vicuna benchmark:
Paper: arxiv.org/abs/2305.14314
Code+Demo: github.com/artidoro/qlora
Samples: colab.research.google.com/dr…
Colab: colab.research.google.com/dr…
Qingqing Cao retweeted
Looking for valuable insights and carefully curated pointers on efficient NLP research directions? 🔎 Check out our updated survey covering an extensive range of topics in the classic NLP pipeline 🚀:
arxiv.org/abs/2209.00099
Product of an amazing collaborative team effort! 🤝
Efficient NLP methods - an up-to-date survey, to appear in TACL. We cover efficiency wrt:
* Data
* Model design
* Pre-training
* Fine-tuning
* Inference & compression
* Hardware utilization
* Evaluation
* Model selection
This was a blast to co-produce!
arxiv.org/abs/2209.00099
Qingqing Cao retweeted
How can NLP be more efficient? We can't blindly scale forever. This survey I'm lucky to be part of presents current efficiency research at many stages of the NLP process, and actionable advice to practitioners for making their NLP more efficient.
arxiv.org/abs/2209.00099 #nlproc
Qingqing Cao retweeted
@lal_yash is presenting this in Bavaro 4 right now! Stop by and have a chat @emnlpmeeting #EMNLP2021
IrEne-viz is a platform showcasing energy consumption of transformer models. Built over IrEne (from #ACL2021), it allows for fine-grained & interpretable analysis of models & their components. Joint work by @yash_lal, Reetu Singh, @harsh3vedi, @sysnlp, @aruna__b, @b_niranjan /n
Qingqing Cao retweeted
With @YiTayML, @anuragarnab, @giffmana, and @ashVaswani, we wrote up a paper on "the efficiency misnomer": arxiv.org/abs/2110.12894
TL;DR:
"No single cost indicator is sufficient for making an absolute conclusion when comparing the efficiency of different models".
Qingqing Cao retweeted
IrEne-viz is a platform showcasing energy consumption of transformer models. Built over IrEne (from #ACL2021), it allows for fine-grained & interpretable analysis of models & their components. Joint work by @yash_lal, Reetu Singh, @harsh3vedi, @sysnlp, @aruna__b, @b_niranjan /n
Qingqing Cao retweeted
How much energy does your #transformer consume? What are its bottlenecks? Check out IrEne, a solution for interpretable energy estimation.
Come talk to us in #ACL2021NLP Session 5D
underline.io/events/167/sess…
@sysnlp's work w/ @harsh3vedi, @lal_yash & @aruna__b
Qingqing Cao retweeted
How much energy does your #transformer consume? What are its bottlenecks? Check out IrEne, a solution for interpretable energy estimation.
@aclmeeting 📹:
youtu.be/Rg7zF81xDgw via @YouTube
Paper:
arxiv.org/abs/2106.01199
@sysnlp's work w/ @harsh3vedi @lal_yash & @aruna__b
Qingqing Cao retweeted
Qingqing Cao (@sysnlp) is in the postdoc market and is an expert in the intersection of systems+NLP. If you are looking for a researcher to work on on-device/efficient NLP and energy-efficient ML, he is your guy.
#NLProc'ers: My student Qingqing Cao @sysnlp, whom I coadvise w/ @aruna__b is looking for postdoc positions in #NLProc ∩ #systems.
He is fantastic in {mobile, energy, efficiency} ∩ {NLP, DNN}.
His CV and research statements:
awk.ai/assets/qingqingcao_cv…
awk.ai/assets/research-state…
Qingqing Cao retweeted
#NLProc'ers: My student Qingqing Cao @sysnlp, whom I coadvise w/ @aruna__b is looking for postdoc positions in #NLProc ∩ #systems.
He is fantastic in {mobile, energy, efficiency} ∩ {NLP, DNN}.
His CV and research statements:
awk.ai/assets/qingqingcao_cv…
awk.ai/assets/research-state…
Qingqing Cao retweeted
Check out this new blogpost by Lalith Suresh with practical advice on low-level management of systems research:
lalith.in/2020/09/27/Low-Lev…
This can be of great help to new grad students, particularly folks with limited industry experience.