Really glad I got to be part of this one. Huge credit to @TheyCallMeMr_ for leading the work, and grateful to the whole team!
1/ RLVR has driven big gains in math and code because many outputs admit reliable automatic checks: an answer matches the expected result, or a program passes tests.
But many real tasks are not like that. Code can be functionally correct but qualitatively terrible or a response may satisfy 4 syntactic constraints but fail 1 semantic constraint.
So happy this is out in EACL Findings! 🥳 Loved being able to contribute a bit, big congrats to @_joestacey_ and all the authors! Really nice practical takeaways on robustness for closed-source LLM fine-tuning.
Proud to share this work led by @oliverjbolton 🎉 We introduce SimMerge, a practical way to make model merging more reliable at scale. Paper and results in the thread 👇
Sadly, this echoes what many authors and ACs have been feeling. The peer-review system is under serious strain. We can't keep pretending it's fine.
I am an AC for ICLR 2026. One of the papers in my batch was just withdrawn. The authors wrote a brief response, explaining why the reviewers failed at their job. I agree with most of their comments. The authors gave up. They are fed up. Just like many of us. I understand. We pretend the emperor has clothes, but he is naked.
Here is the final part of their withdrawal notice. I took the liberty to make it public, to highlight that what we are doing with AI conference reviews these last few years is, basically, madness.
---
Comment: We thank the reviewers for their time.
However, upon reading the reviews for our paper, it became immediately apparent that the four "reject" ratings are not based on good-faith academic disagreement, but on a critical failure to read the submitted paper.
The reviews are rife with demonstrably false claims that are directly contradicted by the text. The core justifications for rejection rely on asserting that key components are "missing" when they are explicitly detailed in the manuscript. Some specific examples are (and many are even fake claims).
Claim: Harder tasks like GSM8K are missing.
Fact: GSM8K results are in many tables, like Table 2 (Section 4.2) and Appendix G.
Claim: The method does not use per-layer ranks.
Fact: This is the entire point of our method. The reviewer clearly mistook our method for the baselines. (Section 2, Table 1).
Claim: The GP kernel is not specified.
Fact: It is specified in Appendix E (Table 6).
Claim: There is no ablation of the method's three stages.
Fact: Section 4.4 ("Ablation Study") and Appendix J are dedicated to this.
Reviewers have a fundamental responsibility to read and evaluate the work they are assigned. The nature of these errors is so fundamental, so systemic in overlooking explicit content, that it goes far beyond what "limited time" or "oversight" can explain. This work has gone through several rounds of revision over the last year. In earlier submissions, the paper usually received borderline or weak-accept scores.
Numerous signs strongly suggest that some reviewers are relying entirely on AI tools to automatically generate peer reviews, rather than fulfilling their fundamental responsibility of personally reading and evaluating manuscripts.
We strongly protest this.
This is a gross disrespect to the authors. It is a flagrant desecration of the reviewer's sacred duty. It fundamentally undermines the integrity of the entire peer-review process.
Given that the reviews are not based on the actual content of our paper, we have decided to withdraw the submission.
We leave this comment so that future readers of the OpenReview page are aware that the items described as "missing" are already present in the submitted manuscript. These negative reviews for this submission are factually unsound and do not reflect the content of the paper. We cannot and will not accept an assessment that is not based on the work we actually submitted.
13
1,574
I’m looking forward to working with the next cohort. Don’t miss the deadline!
I’m very excited to be co-organizing this @NeurIPSConf workshop on LLM evaluations!
Evaluating LLMs is a complex and evolving challenge. With this workshop, we hope to bring together diverse perspectives to make real progress.
See the details below:
We are happy to announce our @NeurIPSConf workshop on LLM evaluations!
Mastering LLM evaluation is no longer optional -- it's fundamental to building reliable models. We'll tackle the field's most pressing evaluation challenges.
For details: sites.google.com/corp/view/l…. 1/3
Happy to have contributed to this work led by @_joestacey_, where we explore improving the robustness of closed-source LLMs fine-tuned on NLI under realistic data constraints.
Our new Aya Vision report is out! 🚀
Big congratulations to @TheyCallMeMr_, @YiyangNan, and the fantastic team at @Cohere_Labs @cohere! 🎉
📄 Technical report: arxiv.org/abs/2505.08751
🛠️ Models and datasets: huggingface.co/collections/C…
When a benchmark becomes the standard, it naturally becomes a target. The real challenge is keeping it robust — which is exactly what we explore in our latest work:
📄 arxiv.org/pdf/2504.20879
It is critical for scientific integrity that we trust our measure of progress.
The @lmarena_ai has become the go-to evaluation for AI progress.
Our release today demonstrates the difficulty in maintaining fair evaluations on @lmarena_ai, despite best intentions.
Have you had a chance to check out Aya Expanse on @huggingface Space yet? 👀
We're thrilled to see that Aya Expanse is trending as one of the @huggingface spaces of the week! 🔥
Check out the Aya Expanse Hugging Face Space to chat, speak, and visualize with Aya in 23 languages. ✍️🗣️🎨huggingface.co/spaces/Cohere…
I can't imagine a better way to kick off my X journey than by sharing the 🌿Aya Expanse 🌿, a state-of-the-art multilingual model family that spans 23 languages.
To learn how Aya Expanse is built: huggingface.co/blog/aya-expa…
Introducing ✨Aya Expanse ✨ – an open-weights state-of-art family of models to help close the language gap with AI.
Aya Expanse is both global and local. Driven by a multi-year commitment to multilingual research.
cohere.com/research/aya
It was a lot of work, but I enjoyed every moment of collaborating with the brilliant people at @CohereForAI, @cohere, and the amazing research community 💙
1
11
201














