Hayate Iso (@iso_map) profile picture
Santa Clara, CA
Joined September 2023
Photo by Hayate Iso (@iso_map)Photo by Hayate Iso (@iso_map)Photo by Hayate Iso (@iso_map)Photo by Hayate Iso (@iso_map)Photo by Hayate Iso (@iso_map)Photo by Hayate Iso (@iso_map)Photo by Hayate Iso (@iso_map)Photo by Hayate Iso (@iso_map)
Hayate Iso retweeted
Checkout how @vllm_project accelerate NeMo-RL rollout ⬇️
RL post-training is hitting a rollout bottleneck. This new paper from #NVIDIAResearch shows how speculative decoding in NeMo-RL + @vllm_project can accelerate rollouts losslessly, with 1.8x higher throughput at 8B and projected 2.5x end-to-end speedup at 235B. Read the full paper: nvda.ws/49kX9eo
1
9
1
33
5,518
Hayate Iso retweeted
Announcing NVIDIA Nemotron 3 Super! 💚120B-12A Hybrid SSM Latent MoE, designed for Blackwell 💚36 on AAIndex v4 💚up to 2.2X faster than GPT-OSS-120B in FP4 💚Open data, open recipe, open weights Models, Tech report, etc. here: research.nvidia.com/labs/nem… And yes, Ultra is coming!
61
203
49
1,201
212,126
What if you could ask a chatbot a question the size of an entire encyclopedia—and get an answer in real time? Multi-million token queries with 32x more users are now possible with Helix Parallelism, an innovation by #NVIDIAResearch that drives inference at huge scale. 🔗 nvda.ws/4eCXxqh
4
28
8
141
17,094
Inference at scale is pushing the boundaries of what today’s systems can handle. One promising direction? #DisaggregatedInference — splitting the serving pipeline into distinct stages like prefill and decode — to unlock better performance across the throughput-interactivity spectrum. While the concept has gained traction and sparked a wave of open source experimentation, real-world deployments remain rare. Why? Because the optimization space is vast, and orchestrating system-level coordination is incredibly complex. 📄 In our latest #NVIDIAResearch, we present a comprehensive systematic study of disaggregated inference at scale, evaluating hundreds of thousands of design points across varied model sizes, traffic patterns, and hardware configurations. Key findings: 1️⃣ Disaggregation shines in prefill-heavy traffic patterns and with larger models. 2️⃣ Dynamic rate matching and elastic scaling are essential to achieving Pareto-optimal performance. 3️⃣ There's no one-size-fits-all architecture — workload-aware tuning is critical. Whether you're building production inference infrastructure or exploring #LLM optimization techniques, this work offers actionable insights to balance system throughput with low-latency responsiveness. 🔬 Read the paper and dive into the data — let’s shape the future of scalable #AI serving, together. 📗research.nvidia.com/publicat…
4
22
2,385
👀New #NVIDIAResearch on boosting MoE model performance with disaggregated serving. Learn how our NVIDIA Dynamo and GB200 NVL72 work together to boost the performance of #AI data centers running MOE models like DeepSeek R1 and the new Llama 4. ⚡ Technical deep dive➡️ nvda.ws/45eElfW
7
15
2
58
5,411
The CFP of DAIS@ICDE2025 has been released 👇
The first call for papers of DAIS 2025 is out. 👉Submission: January 20, 2025, 11:59pm Pacific Time 👉Notification: February 20, 2025 👉Camera-ready due: March 20, 2025 Please visit the website for more details: dais-workshop-icde.github.io…
3
4
794
📢Excited to bring the DAIS workshop to ICDE'25 (w/ @SainyamGalhotra @FarihaAnna @MikeCafarella @sairamgv) The focus is on the emerging idea of compound AI systems with a specific emphasis on data discovery, interactions w/ data, architectures for #AgenticAI+#LLM, and evaluation.
We are excited to announce the First Workshop on Data-AI Systems@ICDE 2025 (DAIS). The workshop will focus on exploring innovative approaches towards building data-aware compound AI systems in the #LLM era. More details: dais-workshop-icde.github.io… #DataManagement #AgenticAI
5
9
1,030
⚡️Are you working with #NLP or #AI-driven products for Natural Language Generation (#NLG)? Task ambiguity is a common pain point and AmbigNLG changes that! AmbigNLG is designed to solve task ambiguity in instructions for NLG. What it is... 👇🧵
1
2
5
599
🎉 Our long paper has been accepted for the #EMNLP2024 main conference! In "AmbigNLG," co-authored with @iso_map, we tackle task ambiguity in NLG instructions to better align LLM outputs with your expectations. 📃 Read more: arxiv.org/abs/2402.17717
5
10
121
9,653
Let’s push the boundaries of #LLMs in text editing tasks! XATU is the first-of-its-kind benchmark that addresses the nuances of text editing, revolutionizing how we edit text with large language models. Don't miss the presentation at #Coling2024! #AI #NLP megagon.ai/xatu-fine-grain-i…
1
4
8
2,026
📢New Preprint📢 LLMs excel at ranking items for retrieval/recommender systems. But, what if we reduce the number of items and instead enforce multiple conditions as ranking instructions? It turns out, LLMs have a long way to go. See our new paper: arxiv.org/abs/2404.00211. 1/n
2
5
2
28
3,872
Hayate Iso retweeted
The dataset proposed in our NAACL paper has been released! Let's analyze the capability of LLMs in terms of the fact-level popularity📈 github: github.com/megagonlabs/witqa #NAACL
📢Accepted to #NAACL2024 main conference🎉 Want to evaluate the ability of LLMs from viewpoints of fact-centric analysis?🤔 We construct a new dataset, WiTQA, including questions with various popularity levels and present a comprehensive analysis. 📖: arxiv.org/abs/2402.13492
5
1
16
3,177
✨ We summarized our #LLM summary evaluation tasks research, which lowers computational costs and solves the Lost-in-the-middle problem. It'll be presented at #EACL2024. If you are attending EACL, you can discuss this with a co-author. 📖 megagon.ai/less-is-more-for-… #ai #nlproc #LLMs
1
4
12
1,568
Hayate Iso retweeted
Frontier models all have at least 100k context length, Gemini 1.5 has even 1m context. What about research and open source? Introducing Long Context Data Engineering, a data driven method achieving the first 128k context open source model matching GPT4-level Needle in a Haystack
8
65
7
452
85,253
📢 We invite authors of accepted EACL Findings papers, without allocated presentation slots, to present their work at the NLP4HR workshop at #EACL24. Deadline: February 1st, 2024, 11:59 pm AoE Find the application on our website: megagon.ai/nlp4hr-2024/ #MachineLearning #NLProc
2
4
679
Join us as we chat with our talented summer interns. Get a glimpse of what our interns worked on and their experience at Megagon Labs. Read our Megagon Summer 2023 Internship Experience article: megagon.ai/megagon-summer-20…
3
7
1,234