Manuel (@Manuel_NLP) header
Manuel (@Manuel_NLP) profile picture

NLP: Text Embeddings and Evaluation/Benchmarking,

VS Code
Joined March 2012
Photo by Manuel (@Manuel_NLP)Photo by Manuel (@Manuel_NLP)Photo by Manuel (@Manuel_NLP)Photo by Manuel (@Manuel_NLP)Photo by Manuel (@Manuel_NLP)Photo by Manuel (@Manuel_NLP)Photo by Manuel (@Manuel_NLP)Photo by Manuel (@Manuel_NLP)Photo by Manuel (@Manuel_NLP)Photo by Manuel (@Manuel_NLP)
📢 ARR August 2026 cycle statistics are available! Did you know ARR publishes statistics for every cycle, including Great Reviewers recognized by Area Chairs? 👏 Many thanks to all reviewers, ACs, SACs, and everyone who contributed! 🔗 stats.aclrollingreview.org/i… #ARR #NLProc
6
43
5,807
CALL FOR AI DETECTION BENCHMARKS I believe it is vital that third parties are able to audit AI detectors like Pangram. Our largest sample size benchmark, from UChicago, was run on Pangram 3 and is a year old now. New LLMs are being released all the time, and the binary task doesn't capture the full spectrum of real world performance. A great benchmark will help shape how the world interacts with AI over the coming years. I am putting out a call for new third-party evaluations that attempt to answer one or more of the following questions: - How well do current detectors catch text from the latest frontier AI models? What about in high context settings, like deep research or an agent harness? - How well do current detectors do on mixed authorship text? Ideally, research in this area explores the full spectrum of AI assistance from light grammatical edits to full co-authorship. - Any sort of sliced evals around language, background, or ethnicity. Null results that show no bias are still interesting to us. What I value in a benchmark is extraordinarily clean data, frontier LLM data, and a high enough difficulty that the benchmark is not saturated by most detectors. If you want credits to run your benchmark, please apply here: pangram.com/contact-us/resea…
36
49
9
528
60,138
Keynote 1 at #EMNLP2026 begins at 9:30. Pascale Fung (@pascalefung) takes the stage in five minutes.
1
23
2,276
EMNLP welcome reception will be in around 1.5 hours! Registration is very well organised and no queue. Volunteers are here to help you on arrival! Enjoy the conference! #EMNLP2026
3
40
3,182
Registration for #EMNLP2026 opens at 14:00 at HUNGEXPO, Budapest, and runs until 19:30. The Welcome Reception follows from 18:00 to 20:00. Program: 2026.emnlp.org/program/
2
22
1,900
🚨 New Paper! 🚨 Multiple-choice benchmarks use accuracy (boring), but education has way cooler scoring schemes! 🤩 👎 Lose points if wrong 🔁 Try until you're right 🃏 Wager points At #EMNLP2026, we show these schemes shift LLM ranks, align with LLM Arena, + unmask LLM quirks
5
21
1
96
3,622
What is TMLR doing under a 300% increase in submissions? - Suspensions of >150 authors - Publicly naming major violations - Submission quotas distinguishing slop patterns - Desk rejections 6% → 53%; focus on clarity - AI reviews evaluating soundness blog.tmlr.org/2026/adapting-…
4
82
19
641
72,343
Emails sent to [email protected] are often quite rude. Before pressing send, please keep in mind that ARR is run mostly by volunteers. PLEASE TREAT US WITH RESPECT! #ARR @ReviewAcl
We have just implemented a self-assessment checker for this and now deployed at aclrollingreview.org/qualifi….
7
1
26
2,773
🚨EvalEval is excited to announce that we are hosting a microconference (hybrid) on Embedding Evaluators in Practice in SF! More details on the agenda and outcomes in thread 🧵 Invite-only, sign up 👇 luma.com/nfu166a1
1
6
1
21
1,947
📢 ARR Oct authors At least one author must have an OpenReview account by the ddl. Others have 1 week to complete profiles. ARR cannot resolve account issues. FAQ on lottery exemptions: aclrollingreview.org/sustain… Only EMNLP/AACL26 can be verified before indexing. #ARR #NLProc
1
5
1
24
4,763
📢 ARR Reviewer Qualifications Checker is Live! For the Oct cycle, each submission must name a qualified service contributor to be guaranteed review. Before submission, you can quickly verify your status: openreview.net/group?id=aclw… Policy: aclrollingreview.org/sustain… #ARR #NLProc
2
11
55
9,146
Manuel retweeted
🔔CFP for the #ACL2027NLP main conference is up! 2027.aclweb.org/calls/main/ 📝Submission deadline to ARR is Jan 4, 2027. ⭐️ACL2027 Theme Track is "Homogenization and knowledge collapse in LLMs".
1
27
3
139
12,607
Manuel retweeted
We're expanding our approach to content provenance to include text in response to EU regulatory requirements, while recognizing the significant limitations of current text watermarking technology. Our tools already help verify whether an image or audio file was created with our models. This work builds on those efforts to help people better understand when content may have been generated or edited with an OpenAI model. In the EU, we’ll start watermarking eligible text from ChatGPT and Codex over the coming weeks to comply with the EU AI Act. Customers using our API can turn on text watermarking for select models worldwide today.
1,003
708
868
10,790
8,045,055
Small bird, fast wings, Kolibri is here. 78B parameters. 3.46B active. Up to 1M tokens of context. Built in Europe. Now the weights are yours. Run it on your own hardware, under Apache 2.0.
435
1,008
490
9,791
2,696,284
Our GPT-6 Pro messages drops from 200 to 100 per week as well, by the way.
64
33
33
568
197,541
RIP Amazon MTurk - I enjoyed working with your for years!
107
Manuel retweeted
arXiv has updated our policy on rate limiting for all submitters. This update was made to fairly distribute moderator time & support the arXiv community of staff, volunteers, readers & authors. Please read our announcement to learn more: blog.arxiv.org/2026/10/01/up…
78
494
240
1,721
764,399
📢 ARR October 2026 authors You do not need to wait until the deadline. Once you make a submission, even a partial one, the designated service contributor form and Responsible Checklist become available. Please complete them within 48h. aclrollingreview.org/sustain… #ARR #NLProc
1
9
1
66
13,494
Who’s responsible for rebranding classifiers as “decision models” ??
19
19
3
310
19,812
Manuel retweeted
OpenReview has informed us that they're experiencing technical difficulties with the ICLR submission site, and that it has been temporarily disabled. They are working on fixing it and we'll make another announcement as soon as it's back online.
15
16
8
366
116,193