Assistant Professor @FollowStevens (2024-) PhD @UofT, @VectorInst Areas: #NLProc #Explainable #AI
Hoboken, New Jersey
Joined January 2014
- Tweets354
- Following613
- Followers851
- Likes1K
If you are at #EMNLP 2026 please check out our three papers at the Main Conference and Findings!
Our papers tackle problems across LLM evaluation, text provenance, and intermediate reasoning:
Using a medical benchmark, we show how explicit reasoning-path supervision can improve reasoning fidelity.
With Zhishan Yuan, Tiago Mateus Moreira de Almeida, Shudong Hao, Nickolas Elias Dawlabani, Ghazia Pervaiz, Yue Ning aclanthology.org/2026.findin…
Zining Zhu retweeted
Brilliant idea! Next up: Apple randomly reboots your Mac if you're building competing tech, Gmail silently edits your email if you mention rival platforms, and Tesla Autopilot swerves if it detects you're working on self-driving cars.
All in the name of safety, of course. Because malicious actors controlling the world’s operating systems, inboxes and cars would be extremely dangerous!
One single score can almost never tell {estimated AI participation extent, type of AI works, detector confidence, etc.} at the same time. Humans using predictor outputs for decisions should be more careful.
NeurIPS 2026 just desk-rejected hundreds of papers because an AI detector said they were AI-written (blog.neurips.cc/2026/06/02/a…).
I understand why they did it. Reviewer time is scarce, and a flood of slop submissions is a real threat to peer review.
But here's where it gets tricky. The Position Paper Track requires papers be "substantially human-written," so I ran one of my own through the same detector they used. It flagged 73% as AI-written — on pretty weak evidence (see screenshots).
The funny part? It's actually closer to 99%. I only hand-edited a few sentences. But the idea, the literature review, the positioning, the data analysis ... all human. The AI wrote the text, across many rounds of back-and-forth with me.
It's like coding: most code today is generated almost entirely by AI. Do we "reject" those systems too?
To me, whether the words were typed by a human matters less than the substance behind them.
What do you think?
PS. I ran this very post through the detector too. It came back 100% AI-generated — even though it took me almost 30 minutes to write, and I think it carries my thinking and is worth reading.
#AI #PeerReview #NeurIPS #AIDetection #AcademicResearch #MachineLearning #ResearchIntegrity #AIEthics #ScholarlyPublishing #AIWriting
Zining Zhu retweeted
Qwen first release on interpretability (qwen scope) is very interesting
they use SAE features to identify what causes repetition in model outputs, then use steering to manufacture a "bad" rollout where the model repeats a lot. this gives RL a clear negative signal to learn from, since repetition barely shows up in normal rollouts so the model never gets punished for it
they also use SAE features as a fingerprint for benchmarks, you look at which features each benchmark activates and compare overlap. lets you find redundancy inside a benchmark and across benchmarks without running any model. for instance 63% of GSM8K features are in MATH but only 10% the other way
Zining Zhu retweeted
AutoResearch saves 90% time for research solution.
But in research, 90% time is spent for the right question.
Zining Zhu retweeted
Announcing new ARENA material: 8 new exercise sets on alignment science, interpretability & AI safety - each containing 1-2 days of structured, hands-on content replicating key papers in the field.
All open source on a public GitHub, and available for study. Here's what's in it:
Zining Zhu retweeted
This is absolutely shameful. Agents of a federal agency unnecessarily escalating, and then executing a defenseless citizen whose offense appears to be using his cell phone camera. Every person regardless of political affiliation should be denouncing this.
Zining Zhu retweeted
TIL that ACL 2026's theme track is "Explainability of NLP Models"! 😮🤩 @aclmeeting
Zining Zhu retweeted
❌ LMs can’t express all next-token distributions — embedding-space limits constrain what’s possible.
🤔 But have you wondered which ones are hardest to elicit?
Our #EMNLP2025 paper finds medium-entropy distributions (without outliers) are the toughest.
🧩arxiv.org/abs/2505.12244
Zining Zhu retweeted
#COLM2025 Our last invited talk if you are still around: Prof. Zining Zhu is presenting "Improving LLM reasoning with mechanistic insights" @zhuzining
Zining Zhu retweeted
New paper: What happens when an LLM reasons?
We created methods to interpret reasoning steps & their connections: resampling CoT, attention analysis, & suppressing attention
We discover thought anchors: key steps shaping everything else. Check our tool & unpack CoT yourself 🧵
Reviewers should perhaps be prohibited from changing the scores they give on the day of seeing the scores of their own papers submitted to @ReviewAcl.
Zining Zhu retweeted
🚨Deadline alert: If you work on LLM explainability for reasoning and planning, submit your work by June 23!
- Non-archival, two formats (long/short)
- Welcome recently accepted papers and dual submissions
- 🏆Two awards will be announced!
Details: xllm-reasoning-planning-work…
