Eval Sys (@EvalSysOrg) profile picture

EVAL SYS is a continuously evolving, open-source community! We are on a mission to track and advance model agentic capabilities.

Joined August 2025
MCPMark Verified is live ✅ Independently benchmarked, reproducible MCP agent performance. And the first Verified result is in 🎉 Congrats @Kimi_Moonshot K2.7-Code — Ranked #2 at 81.1% +8.3pts over K2.6 · Ahead of Claude Opus 4.8 max Open-source code models are closing in 💪
🌘 Kimi-K2.7-Code, our latest coding model, is now released and open-sourced! 🔷 Improved coding & agent performance over K2.6: +21.8% on Kimi Code Bench v2, +11.0% on Program Bench, and +31.5% on MLS Bench Lite. 🔷 Reasoning efficiency: Less overthinking, with 30% lower reasoning-token usage compared to K2.6. 🔷 Long-horizon coding: Improved instruction following, higher end-to-end coding task success rates. ⚡️ 6x High-Speed Mode coming soon! 🔌 Available today via Kimi API and Kimi Code. 🔗 Kimi Code: kimi.com/code 🔗 API: platform.moonshot.ai
2
1
2
8
7,955
MCPMark Leaderboard Update 🚀 🌟 DeepSeek-V3.2-thinking jumps to the #1 spot among open-source models — and we’re honored to see MCPMark cited in the @deepseek_ai technical report. ⚡️ Gemini 3 Pro High @GoogleDeepMind now leads with the highest pass@1 and pass@4 success rates. This update brings two newly released models onto the leaderboard: Gemini 3 and DeepSeek V3-2.
2
8
1
12
1,594
We're always looking for contributors in community! Check out more detail here: Github:github.com/eval-sys/mcpmark Website:mcpmark.ai
4
2,133
MCPMark Leaderboard Update 🚀 🌟 Qwen-3-Coder takes the #1 spot among open-source models, with an impressive per-run cost of just $36.46. ⚡️ Grok-Code-Fast-1 delivers the lowest per-run cost ($16.08) and the fastest average agent time (156.63s) across the top 10 models. Kimi-K2-0905 outperforms Kimi2 in success rate, though at nearly double the per-run cost and average agent time. Notably, Qwen-3-Coder achieves a success rate close to O3, but at roughly one-third the per-run cost — offering the community a highly cost-effective option for MCP tool-use applications. This update introduces three newly released models to the leaderboard: Qwen-3-Max, Grok-Code-Fast-1, and Kimi-K2-0905.
5
21
8
132
95,374
🚀 MCPMark website updated! → mcpmark.ai/ On MCPMark, you can now dive into each task’s description, verification, and model performance. We’ve also added model trajectories to each task leaderboard, so you can clearly see how models execute step by step. Many users found this super helpful for understanding models better—some even spent the whole afternoon exploring it. 💡 Feedback is always welcome—stay tuned for more updates! 👍 Big thanks to @arvinxu95 for the brilliant UX design and feature delivery!
1
4
1,608