Robbie Jones retweeted
Why can contrastive pretraining on *unlabeled data* improve robustness to distribution shift? (It's not about domain invariance!) Come to our ICML Oral at 2:05pm - 2:25pm in Ballroom 1 & 2, and our poster session at Hall E, Poster 317, to find out more!
Pretraining is ≈SoTA for domain adaptation: just do contrastive learning on *all* unlabeled data + finetune on source labels. Features are NOT domain-invariant, but disentangle class & domain info to enable transfer. Theory & exps: arxiv.org/abs/2204.00570 arxiv.org/abs/2204.02683
Robbie Jones retweeted
We’ll be presenting WILDS v2.0 as an oral at ICLR! We extended the WILDS benchmark of real-world shifts by adding unlabeled data, which can be used for domain adaptation and representation learning.
Talk + poster: iclr.cc/virtual/2022/oral/69…
Paper: arxiv.org/abs/2112.05090
🧵
Robbie Jones retweeted
Pretraining is ≈SoTA for domain adaptation: just do contrastive learning on *all* unlabeled data + finetune on source labels. Features are NOT domain-invariant, but disentangle class & domain info to enable transfer. Theory & exps: arxiv.org/abs/2204.00570 arxiv.org/abs/2204.02683
Robbie Jones retweeted
Just saw this tweet thread about our fine-tuning paper - I love it! Their explanation is so easy to understand.
Today, we're looking at fine-tuning large models, and this paper submitted to ICLR: openreview.net/forum?id=UYne…
It shows fine-tuning can hurt performance on out-of-distribution examples, and explains how using some nice theory. We'll be keeping an eye on this! (1/8)
Robbie Jones retweeted
How should you fine-tune a large pretrained model (CLIP, SimCLR) robustly? We find that standard fine-tuning can do poorly out-of-distribution (test data ≠ fine-tuning data). Our analysis leads to a simple fix, higher accuracy on 10 datasets. arxiv.org/abs/2202.10054 (ICLR Oral)
Robbie Jones retweeted
This appears in #ICLR2021. Please check out our paper, videos, poster, code, etc!
ICLR poster link: iclr.cc/virtual/2021/poster/…
ArXiv: arxiv.org/abs/2012.04550
Codalab: worksheets.codalab.org/works…
Github: github.com/p-lambda/in-n-out
🍔🍟"In-N-Out: Pre-Training and Self-Training using Auxiliary Information for Out-of-Distribution Robustness"
arxiv.org/abs/2012.04550
Real-world tasks (crop yield prediction from satellites) are often label-scarce. Only some countries have labels - how do we generalize globally?
Robbie Jones retweeted
🍔🍟"In-N-Out: Pre-Training and Self-Training using Auxiliary Information for Out-of-Distribution Robustness"
arxiv.org/abs/2012.04550
Real-world tasks (crop yield prediction from satellites) are often label-scarce. Only some countries have labels - how do we generalize globally?











