Jenny (Xuchan) Bao retweeted
We published a new version of our Emergent Misalignment paper in Nature!
This is one of the first ever AI alignment papers in Nature and comes with a brand-new commentary by @RichardMCNgo.
Here's the story of EM over the last year 🧵
Jenny (Xuchan) Bao retweeted
Our new paper: Emergent misalignment extends to *reasoning* LLMs.
Training on narrow harmful tasks causes broad misalignment.
Reasoning models sometimes resist being shut down and plot deception against users in their chain-of-thought (despite no such training)🧵
Jenny (Xuchan) Bao retweeted
Surprising new results:
We finetuned GPT4o on a narrow task of writing insecure code without warning the user.
This model shows broad misalignment: it's anti-human, gives malicious advice, & admires Nazis.
This is *emergent misalignment* & we cannot fully explain it 🧵
Jenny (Xuchan) Bao retweeted
New paper:
We train LLMs on a particular behavior, e.g. always choosing risky options in economic decisions.
They can *describe* their new behavior, despite no explicit mentions in the training data.
So LLMs have a form of intuitive self-awareness 🧵
Jenny (Xuchan) Bao retweeted
New paper:
Are LLMs capable of introspection, i.e. special access to their own inner states?
Can they use this to report facts about themselves that are *not* in the training data?
Yes — in simple tasks at least! This has implications for interpretability + moral status of AI 🧵
Jenny (Xuchan) Bao retweeted
🆕📜We study large language models’ ability to extrapolate to longer problems!
1) finetuning (with and without scratchpad) fails
2) few-shot scratchpad confers significant improvements
3) Many more findings (see the table & thread)
Paper: [arxiv.org/abs/2207.04901]
1/
Jenny (Xuchan) Bao retweeted
I am looking for incoming students at NYU Courant CS or Center for Data Sci starting fall 2022. If you have interest in working with me on topics like meta/continual/embodied/representation learning+vision, please reach out & mention me in your PhD application. Deadline is Dec 12
Jenny (Xuchan) Bao retweeted
Linear interpolation from initial to final neural net params typically decreases the loss monotonically
We investigate this phenomenon empirically and theoretically in new preprint arxiv.org/abs/2104.11044
w/ @juhan_bae, @michaelrzhang, @stanislavfort, Rich Zemel, @RogerGrosse
Jenny (Xuchan) Bao retweeted
Replying to @RogerGrosse
@RogerGrosse (my adviser) is teaching a course on Neural Net Training Dynamics this semester @UofT . Cannot recommend enough! cs.toronto.edu/~rgrosse/cour…
In particular, the Taylor series notes explain nicely some analysis in @XuchanB, mine and Roger's paper on regularized LAEs.
Jenny (Xuchan) Bao retweeted
Excited to share our work extending @LaurentLessard's IQC framework to the smooth games setting. IQCs are a surprisingly powerful tool, and let us quickly analyze global convergence of lots of algorithms just by building and solving SDPs.
By @Guodzh, w/ @XuchanB, and LL.
New paper alert: arxiv.org/pdf/2009.11359.pdf
We provide a unified and automated method to analyze first-order methods for smooth & strongly-monotone games. The convergence rate for any first-order method can be obtained via a mechanical procedure of deriving and solving an SDP.
Jenny (Xuchan) Bao retweeted
New paper alert: arxiv.org/pdf/2009.11359.pdf
We provide a unified and automated method to analyze first-order methods for smooth & strongly-monotone games. The convergence rate for any first-order method can be obtained via a mechanical procedure of deriving and solving an SDP.
Jenny (Xuchan) Bao retweeted
With the right regularizer, linear autoencoders recover the ordered principal components, but very slowly. Interesting model system for representation learning since we know the optimal representation.
New work w/ @XuchanB, @james_r_lucas, and @sushnt. arxiv.org/abs/2007.06731
Jenny (Xuchan) Bao retweeted
TimbreTron: musical timbre transfer using "image" domain style transfer on time-frequency representations, with high-quality waveform output. New paper with @osageev and a team of amazing UofT undergrads.
youtube.com/watch?v=YQAupr7J…



















