Jenny (Xuchan) Bao (@XuchanB) profile picture

Machine learning PhD student @ UofT

Joined December 2014
Photo by Jenny (Xuchan) Bao (@XuchanB)Photo by Jenny (Xuchan) Bao (@XuchanB)
Jenny (Xuchan) Bao retweeted
We published a new version of our Emergent Misalignment paper in Nature! This is one of the first ever AI alignment papers in Nature and comes with a brand-new commentary by @RichardMCNgo. Here's the story of EM over the last year 🧵
23
116
21
695
93,460
Jenny (Xuchan) Bao retweeted
Our new paper: Emergent misalignment extends to *reasoning* LLMs. Training on narrow harmful tasks causes broad misalignment. Reasoning models sometimes resist being shut down and plot deception against users in their chain-of-thought (despite no such training)🧵
27
56
12
303
32,432
Jenny (Xuchan) Bao retweeted
Surprising new results: We finetuned GPT4o on a narrow task of writing insecure code without warning the user. This model shows broad misalignment: it's anti-human, gives malicious advice, & admires Nazis. 
This is *emergent misalignment* & we cannot fully explain it 🧵
424
922
819
6,676
1,933,687
Jenny (Xuchan) Bao retweeted
New paper: We train LLMs on a particular behavior, e.g. always choosing risky options in economic decisions. They can *describe* their new behavior, despite no explicit mentions in the training data. So LLMs have a form of intuitive self-awareness 🧵
46
150
64
943
156,242
Jenny (Xuchan) Bao retweeted
New paper: Are LLMs capable of introspection, i.e. special access to their own inner states? Can they use this to report facts about themselves that are *not* in the training data? Yes — in simple tasks at least! This has implications for interpretability + moral status of AI 🧵
25
82
38
528
151,176
Jenny (Xuchan) Bao retweeted
🆕📜We study large language models’ ability to extrapolate to longer problems! 1) finetuning (with and without scratchpad) fails 2) few-shot scratchpad confers significant improvements 3) Many more findings (see the table & thread) Paper: [arxiv.org/abs/2207.04901] 1/
4
38
11
229
Jenny (Xuchan) Bao retweeted
I am looking for incoming students at NYU Courant CS or Center for Data Sci starting fall 2022. If you have interest in working with me on topics like meta/continual/embodied/representation learning+vision, please reach out & mention me in your PhD application. Deadline is Dec 12
14
73
8
335
Jenny (Xuchan) Bao retweeted
Linear interpolation from initial to final neural net params typically decreases the loss monotonically We investigate this phenomenon empirically and theoretically in new preprint arxiv.org/abs/2104.11044 w/ @juhan_bae, @michaelrzhang, @stanislavfort, Rich Zemel, @RogerGrosse
2
43
6
231
Jenny (Xuchan) Bao retweeted
Replying to @RogerGrosse
@RogerGrosse (my adviser) is teaching a course on Neural Net Training Dynamics this semester @UofT . Cannot recommend enough! cs.toronto.edu/~rgrosse/cour… In particular, the Taylor series notes explain nicely some analysis in @XuchanB, mine and Roger's paper on regularized LAEs.
8
28
2
155
Jenny (Xuchan) Bao retweeted
Excited to share our work extending @LaurentLessard's IQC framework to the smooth games setting. IQCs are a surprisingly powerful tool, and let us quickly analyze global convergence of lots of algorithms just by building and solving SDPs. By @Guodzh, w/ @XuchanB, and LL.
New paper alert: arxiv.org/pdf/2009.11359.pdf We provide a unified and automated method to analyze first-order methods for smooth & strongly-monotone games. The convergence rate for any first-order method can be obtained via a mechanical procedure of deriving and solving an SDP.
4
15
Jenny (Xuchan) Bao retweeted
New paper alert: arxiv.org/pdf/2009.11359.pdf We provide a unified and automated method to analyze first-order methods for smooth & strongly-monotone games. The convergence rate for any first-order method can be obtained via a mechanical procedure of deriving and solving an SDP.
2
8
2
47
Jenny (Xuchan) Bao retweeted
With the right regularizer, linear autoencoders recover the ordered principal components, but very slowly. Interesting model system for representation learning since we know the optimal representation. New work w/ @XuchanB, @james_r_lucas, and @sushnt. arxiv.org/abs/2007.06731
3
30
131
Jenny (Xuchan) Bao retweeted
TimbreTron: musical timbre transfer using "image" domain style transfer on time-frequency representations, with high-quality waveform output. New paper with @osageev and a team of amazing UofT undergrads. youtube.com/watch?v=YQAupr7J…
3
39
3
103