PhD @RiceCompSci, Previously Master @JHUCompSci
Baltimore, MD
Joined August 2023
- Tweets40
- Following85
- Followers71
- Likes97
#EMNLP2026 is coming, and GR-SAP is accepted to the Main Conference!
Fine-tuning an aligned LLM, even on benign data, can quietly erode its safety 😬
The classic fix: mix the original safety alignment data back in. But it’s rarely released.
Can the model synthesize its own? 🤔
✨ Synthesize: the aligned model generate its own domain-specific safety data, a reliable proxy for the original alignment data (theory + experiments)
🔒 Preserve: mixing in just 10% consistently reduces safety degradation across 4 model families & 5 downstream tasks
1
3
114
🎤 Virtual live talk: Sun, Oct 25, 18:00 CET on Zoom
📄 arxiv.org/abs/2603.10243
💻 github.com/chili-lab/GR-SAP
📊 docs.google.com/presentation…
w/ @jzhou_jz & @hanjie_chen
It is now accepted to EMNLP main (15.4% acceptance rate this year)! 🥳
Shoutout again to @jzhou_jz and @hanjie_chen for making this possible!
⚠️ Fine-tuning LLMs can easily compromise their safety alignment, even with seemingly harmless data!
🚀 We propose ✨GR-SAP✨ to preserve safety alignment with model-synthesized data during fine-tuning.
Check our new preprint!
📄 arxiv.org/abs/2603.10243
Zhouxiang Fang retweeted
What if LLM steering is more than vector addition?
Excited to share that our paper “Spherical Steering: Geometry-Aware Activation Rotation for Language Models” has been accepted to @icmlconf #ICML2026.
We introduce a training-free way to steer LLMs by rotating activations on the hypersphere.
Paper: arxiv.org/abs/2602.08169
Code: github.com/chili-lab/Spheric…
Nice work! We have similar findings in the LLM safety domain! A model could effectively preserve it's safety alignment via generative replay during downstream finetuning!
arxiv.org/abs/2603.10243
How much does a language model forget when finetuned on new tasks? We show both model size and optimization matter and forgetting can be nearly eliminated with self-generated replay!
arxiv.org/abs/2605.26097
w/@mrtnm @dongkyucho @ShikaiQiu @rumichunara @Pavel_Izmailov 1/8
Zhouxiang Fang retweeted
What is the key bottleneck to scaling looped transformers (LT)? A major challenge is their speed: the looped operation is coupled w/ full quadratic attention. More loop, more powerful, but much slower.
Introducing LT2: linear-time looped transformers that loop over linear attention and sparse attention. Linear and sparse attention give the loop speed, making it a fast loop. The loop, in turn, gives linear attention iterative control over its recurrent memory and recursively enlarges the receptive field for sparse attention. Fast attention accelerating the loop, the loop enriching attention, making LT2 a pareto-frontier architecture compared to standard looped transformers.
This is a large paper. We did careful ablations in pretraining to find the best architecture, and we used this architecture to distill a hybrid looped transformer, Ouro-hybrid-1.4B, to deliver both industry-level performance and fast inference speed. To read more:
Paper: arxiv.org/pdf/2605.20670
Code: github.com/chili-lab/LT2
Project: charlesdddd.github.io/lt2/
Model: huggingface.co/chili-lab/Our…
Zhouxiang Fang retweeted
Thrilled to share that SportR is accepted to #ICLR2026! 🏀⚽🏈🏓🏸
Can multimodal LLMs really understand sports — or just describe what they see?
Our answer: not yet. Even state-of-the-art models struggle to reason about fouls and tactics. There's a huge gap between "seeing" and "understanding" sports. We built the benchmark that shows why. 👇
Zhouxiang Fang retweeted
Hello, we are a research team at Northeastern University conducting a paid remote study on human–AI collaboration in coding workflows. We are currently looking for developers to participate in testing these workflows.
Eligibility:
Software engineering experience in software or website development
Familiar with AI-assisted coding tools (e.g., GitHub Copilot, Cursor)
Proficient with GitHub/GitLab, VS Code, and code review workflows
Study Details:
Collaborate with an AI agent to develop and review code (~4 hours in total)
Compensation ($25/hr) via digital gift cards
Participation is entirely voluntary.
Interested? Sign up here: forms.gle/sUuFbyn5KnGZVd1N7
Contact me if you have any questions!
Zhouxiang Fang retweeted
We’re excited to receive the NVIDIA Academic Grant Program Award #NVIDIAGrant🎉🥳
Thank you to @NVIDIAAIDev @NVIDIAAI for the support! This award is extremely helpful in advancing our research on LLM Architectural Design and Optimization, led by @ChunyuanDeng @RiceCompSci🔥
⚠️ Fine-tuning LLMs can easily compromise their safety alignment, even with seemingly harmless data!
🚀 We propose ✨GR-SAP✨ to preserve safety alignment with model-synthesized data during fine-tuning.
Check our new preprint!
📄 arxiv.org/abs/2603.10243
💡 Instead, we synthesize safety alignment data from the model itself!
🎯 Our theoretical and empirical analyses demonstrate it serves as a high-fidelity proxy for the original alignment data.
🏆 Consistently outperforms open-source alternatives in preserving safety alignment!
Preprint: arxiv.org/abs/2603.10243
Huge shoutout to my amazing advisors and collaborators🔥 @jzhou_jz @hanjie_chen
Work done at @RiceUniversity @RiceCompSci
Zhouxiang Fang retweeted
spent the weekend poking at linear attention + different nonlinear key–value distributions.
Little bit surprise to me: once we add norms or short convs to K/V, we’re basically good enough.
Blog: charlesdddd.github.io/blog/l….
Zhouxiang Fang retweeted
📢I'm recruiting a postdoc at Rice University! Looking for someone with a PhD in CS (or a related field) and expertise in LLMs and Trustworthy & Responsible AI.
I'll be at #COLM2025 from 10/8-10/10. Happy to chat in person!
Feel free to email or DM me if you're interested.😀
Our paper is accepted to the main conference of #EMNLP2025! Check the post for more details!
🤔 To what extent can Large Language Models (LLMs) solve unseen tasks via In-context Learning (ICL) ?
💥 We propose a new general framework ✨ ICL ciphers ✨ to quantify ``learning'' in ICL via substitution ciphers.
🔗 :
Check our new preprint!
arxiv.org/abs/2504.19395
Zhouxiang Fang retweeted
Our work on readability evaluation for Plain Language Summarization will appear at #EMNLP2025!! @DanielKhashabi @mdredze
Paper: arxiv.org/pdf/2508.19221
TLDR: Traditional readability metrics correlate poorly with human judgements & LMs consider deeper readability features. 1/6
Zhouxiang Fang retweeted
I’m at #ICML this week !🍁🍁🍁
@hanjie_chen and I will present the work on Wed 4:30 pm poster session (July 16th), feel free to stand by if you are also interested in steering & controlling! 😃
[#ICML2025] If steering vector can be multiplied by a scalar to control its effect, why it should be a vector instead of a steering "region"?
We introduce distribution-wise intervention, a simple method that directly learns the latent intervention region through reparameterization. Given a deterministic intervention function (mean network), we equip it with an independent network (logvar network) to learn its surrounding region.
🎄 Key Takeaway:
- D-Intervention is effective at early layers, achieving a drop-in gain compared to its deterministic counterparts at 2% to 4% improvement.
- The downstream performance is pretty correlated w/ its learned variance, the more it can explore its surrounding, the more it could achieve better results.
- Controlling test-time stochasticity to fulfill the need for creativity in different tasks! Very fun to play with 😀
Zhouxiang Fang retweeted
There have been various efforts on disentangling "task learning" vs "task recall" in LLMs. We've recently explored a fresh angle by borrowing from cryptography: with substitution ciphers, we transform a given task into an equivalent, but cryptic (no pun intended!!) forms. Essentially, we swap some tokens in the input with others, making English sentences harder for humans to read. The twist? If this transformation is "bijective" (reversible), the task stays intact in an abstract sense, despite the changes.
Empirically we find that using "bijective" ciphers of In-Context Learning (ICL) consistently leads to higher performance compared to "non-bijective" ciphers. This strongly indicates that ICL involves "task learning." **However,** the performance gap is small, suggesting that the extent of "task learning" may be more limited than previously thought.
By Zhouxiang Fang @FocusV857 and Aayush Mishra @aamixsh ; with contributions from me, @muhan_gao and @anqi_liu33 @JHUCompSci
Paper link: arxiv.org/abs/2504.19395
