🎉 Our MulTypo paper has been accepted to ACL 2026 (Main Conference)! 📄 arxiv.org/pdf/2510.09536 💻 github.com/cisnlp/multypo LLMs are used in real-world multilingual settings, where users constantly make typos. But almost all benchmarks assume perfect, clean input. 👉 So… how robust are LLMs to realistic typos across languages? We introduce MulTypo, a multilingual typo generation algorithm that simulates human-like typing errors: • grounded in language-specific keyboard layouts • models real typing behavior (not random noise) • validated with human evaluation → resulting typos look much closer to what users actually produce We then evaluate 18 LLMs across various tasks and multiple languages. **Key findings**: 📉 Even small amounts of typos consistently degrade performance 🧠 Tasks requiring generation and reasoning are especially fragile ⚙️ Instruction tuning improves clean performance -- but can make models more brittle under noise 🌍 Robustness varies across languages: • high-resource languages are more stable • translation from English is more robust than into English 💡 Takeaway: LLMs don’t just struggle with adversarial attacks -- they struggle with everyday human mistakes. To support the community, we release MulTypo as a Python package: 👉 Easily inject realistic typos into your data 👉 Benchmark robustness under real-world noise 👉 Build and evaluate more reliable multilingual systems We hope this helps move evaluation beyond clean benchmarks and towards more realistic, user-facing settings 🚀