Postdoctoral fellow with @ETH_AI_Center. CSS + NLP. Previously CS PhD @umdcs, intern at @msftresearch and @ai2_allennlp.
Joined May 2011
- Tweets933
- Following608
- Followers1.3K
- Likes5K
I'll be at #EMNLP2026 --- looking forward to (re)-connecting with folks!
Ben will be presenting our work on the automated reproduction of social science results (poster Tuesday at 9:00)
Can AI agents read a social science paper and write the code from scratch to reproduce its results?
No access to original code. Just text + data.
New paper with Ben Kohler, @david_rzs, @__jae_1, and @miserlis_
👇
Alexander Hoyle retweeted
I'm on the faculty job market for Fall 2027 assistant professorship. I work on the science of AI/NLP evaluation, which is currently undergoing a crisis:
I really enjoyed working on this paper and learned a lot about reasoning models in the process! Hope the paper will provide equally edifying for readers :)
Now accepted to #EMNLP2026!
Can AI agents read a social science paper and write the code from scratch to reproduce its results?
No access to original code. Just text + data.
New paper with Ben Kohler, @david_rzs, @__jae_1, and @miserlis_
👇
Alexander Hoyle retweeted
Missed the #EACL2027 deadline? Don’t worry! You can still submit:
💻 your #NLProc demo papers by 22 September, see 2027.eacl.org/calls/demos/
🏭 your #NLProc industry papers by 11 September, see 2027.eacl.org/calls/industry…
Alexander Hoyle retweeted
Want to do a 𝐩𝐨𝐬𝐭𝐝𝐨𝐜 in my group at 𝐄𝐓𝐇 𝐙𝐮𝐫𝐢𝐜𝐡?
Check out our new position on "Multimodal reasoning models for oncology"
which is centered around a new collab between our lab and kaiko . ai
In addition, our group is engaged with the ETH AI Center and SwissAI projects where we have access to large-scale GPU ressources. Links in comment.
Delighted to announce the next step in my career!
After my postdoc, I will begin a joint appointment at TU Wien and the Complexity Science Hub Vienna as an Assistant Professor in NLP! My deepest thanks to all who helped me along the way
I'm hiring—details on PhD positions below
Why join? Vienna has a vibrant, rapidly-growing AI scene and a terrific quality of life; TU Wien has an excellent CS department; the Complexity Science Hub is a home to ambitious interdisciplinary work.
If you are curious, opinionated, driven: come!
New preprint out!
Recent work tasks LLM agents with re-running existing social science replication code. Given current capabilities, that should be table stakes
Here, we move up a level of abstraction, and ask models to reproduce results from a paper’s descriptions alone
Can AI agents read a social science paper and write the code from scratch to reproduce its results?
No access to original code. Just text + data.
New paper with Ben Kohler, @david_rzs, @__jae_1, and @miserlis_
👇
We view this effort as a first rung on a reproducibility ladder, eventually ending with wholesale replication from only a research question
Ben, and the rest of the team, did terrific work on this and I’m really excited for what’s next!
Now on arXiv: arxiv.org/abs/2604.21965
Thanks for mavhing me! I had a lovely time!
We were thrilled to host @miserlis_ at our lab!
His insightful talk on topic modeling sparked great discussions and fresh perspectives across the team. Thanks for the visit, we hope to welcome you back soon!
#NLProc
![Meme gently critiquing the practice of silicon sampling.
STOP SILICON SAMPLING
LLMS ARE NOT SIMULATIONS OF PEOPLE
DOZENS OF PAPERS yet NO REAL-WORLD USE FOUND for "polling" LLMs
Wanted to prompt an LLM anyway for a laugh? We had a tool for that: It was called "GUESSING"
"Yes, please tell me your presidential preference on a 1-100 scale." "Mark, this is your digital twin" – Statements dreamed up by the utterly Deranged
LOOK at what Prompt Engineers have been demanding your Respect for all this time, with all the layers and attention heads we built for them:
(This are REAL figures, using FAKE people).
[figures from some papers, content not important]
"Hello, pretend you are a Korean-American living in New Jersey, what is your favorite gum brand?"
They have played us for absolute fools](https://pbs.twimg.com/media/HFdvF5vXkAALefL.png?name=orig)
ALT Meme gently critiquing the practice of silicon sampling. STOP SILICON SAMPLING LLMS ARE NOT SIMULATIONS OF PEOPLE DOZENS OF PAPERS yet NO REAL-WORLD USE FOUND for "polling" LLMs Wanted to prompt an LLM anyway for a laugh? We had a tool for that: It was called "GUESSING" "Yes, please tell me your presidential preference on a 1-100 scale." "Mark, this is your digital twin" – Statements dreamed up by the utterly Deranged LOOK at what Prompt Engineers have been demanding your Respect for all this time, with all the layers and attention heads we built for them: (This are REAL figures, using FAKE people). [figures from some papers, content not important] "Hello, pretend you are a Korean-American living in New Jersey, what is your favorite gum brand?" They have played us for absolute fools
I wrote a blog post on my experience using AI for slide generation Basic idea: write your lecture notes first, then prompt the LLM to produce corresponding slides in reveal.js (h/t @ChenhaoTan). I'm picky about my slides but was happy with the results!
(link in thread below)
ALT A slide showing that the posterior is proportional to the likelihood times the prior
alexanderhoyle.com/posts/ai-…
I included some comparisons to other approaches, including Copilot, which helped me understand why so many people hate Copilot
(The screenshot is not cut off. It just looks like that)
ALT Copilot-generated powerpoint slides for bayes' rule: very bad
Alexander Hoyle retweeted
If you, as a scientist, cannot be bothered to engage in the intellectual work of science, please quit your job and leave it to someone with skill and integrity.
OpenAI’s latest product let’s you vibe code science trib.al/kxbfFr0
A new EACL paper! There's been a lot of interest in LLMs for annotation recently, and they tend to treat humans as a ground truth. But we know that's a simplification---humans disagree all the time. Here, we investigate whether we can model that disagreement with LLMs
🚨 Using reasoning LLMs as annotators? You might be erasing critical human disagreements.
Our EACL'25 paper shows RLVR-style reasoning actually HARMS disagreement modeling—even when carefully prompted to consider it! 🙀
📄 Paper: arxiv.org/abs/2506.19467🧵👇
