
Hayate Iso retweeted
Checkout how @vllm_project accelerate NeMo-RL rollout ⬇️
RL post-training is hitting a rollout bottleneck.
This new paper from #NVIDIAResearch shows how speculative decoding in NeMo-RL + @vllm_project can accelerate rollouts losslessly, with 1.8x higher throughput at 8B and projected 2.5x end-to-end speedup at 235B.
Read the full paper: nvda.ws/49kX9eo
Hayate Iso retweeted
Announcing NVIDIA Nemotron 3 Super!
💚120B-12A Hybrid SSM Latent MoE, designed for Blackwell
💚36 on AAIndex v4
💚up to 2.2X faster than GPT-OSS-120B in FP4
💚Open data, open recipe, open weights
Models, Tech report, etc. here:
research.nvidia.com/labs/nem…
And yes, Ultra is coming!
Hayate Iso retweeted
What if you could ask a chatbot a question the size of an entire encyclopedia—and get an answer in real time?
Multi-million token queries with 32x more users are now possible with Helix Parallelism, an innovation by #NVIDIAResearch that drives inference at huge scale.
🔗 nvda.ws/4eCXxqh
Hayate Iso retweeted
Inference at scale is pushing the boundaries of what today’s systems can handle. One promising direction? #DisaggregatedInference — splitting the serving pipeline into distinct stages like prefill and decode — to unlock better performance across the throughput-interactivity spectrum.
While the concept has gained traction and sparked a wave of open source experimentation, real-world deployments remain rare. Why? Because the optimization space is vast, and orchestrating system-level coordination is incredibly complex.
📄 In our latest #NVIDIAResearch, we present a comprehensive systematic study of disaggregated inference at scale, evaluating hundreds of thousands of design points across varied model sizes, traffic patterns, and hardware configurations.
Key findings:
1️⃣ Disaggregation shines in prefill-heavy traffic patterns and with larger models.
2️⃣ Dynamic rate matching and elastic scaling are essential to achieving Pareto-optimal performance.
3️⃣ There's no one-size-fits-all architecture — workload-aware tuning is critical.
Whether you're building production inference infrastructure or exploring #LLM optimization techniques, this work offers actionable insights to balance system throughput with low-latency responsiveness.
🔬 Read the paper and dive into the data — let’s shape the future of scalable #AI serving, together.
📗research.nvidia.com/publicat…
Hayate Iso retweeted
👀New #NVIDIAResearch on boosting MoE model performance with disaggregated serving.
Learn how our NVIDIA Dynamo and GB200 NVL72 work together to boost the performance of #AI data centers running MOE models like DeepSeek R1 and the new Llama 4. ⚡
Technical deep dive➡️ nvda.ws/45eElfW
Hayate Iso retweeted
🚀 Want to improve your LLM responses? Read our tutorial for implementing AmbigNLG! Addressing task ambiguity in Natural Language Generation to drive more accurate, context-aligned outputs. @iso_map
megagon.ai/ambignlg-a-tutori… #AmbigNLG #NLP #tutorial #LLMs #EMNLP2024 #MLEngineering
Hayate Iso retweeted
The CFP of DAIS@ICDE2025 has been released 👇
The first call for papers of DAIS 2025 is out.
👉Submission: January 20, 2025, 11:59pm Pacific Time
👉Notification: February 20, 2025
👉Camera-ready due: March 20, 2025
Please visit the website for more details: dais-workshop-icde.github.io…
Hayate Iso retweeted
📢Excited to bring the DAIS workshop to ICDE'25 (w/ @SainyamGalhotra @FarihaAnna @MikeCafarella @sairamgv) The focus is on the emerging idea of compound AI systems with a specific emphasis on data discovery, interactions w/ data, architectures for #AgenticAI+#LLM, and evaluation.
We are excited to announce the First Workshop on Data-AI Systems@ICDE 2025 (DAIS). The workshop will focus on exploring innovative approaches towards building data-aware compound AI systems in the #LLM era. More details: dais-workshop-icde.github.io…
#DataManagement #AgenticAI
Hayate Iso retweeted
🎉 Our long paper has been accepted for the #EMNLP2024 main conference!
In "AmbigNLG," co-authored with @iso_map, we tackle task ambiguity in NLG instructions to better align LLM outputs with your expectations.
📃 Read more: arxiv.org/abs/2402.17717
Hayate Iso retweeted
Let’s push the boundaries of #LLMs in text editing tasks! XATU is the first-of-its-kind benchmark that addresses the nuances of text editing, revolutionizing how we edit text with large language models. Don't miss the presentation at #Coling2024! #AI #NLP megagon.ai/xatu-fine-grain-i…
Hayate Iso retweeted
📢New Preprint📢
LLMs excel at ranking items for retrieval/recommender systems. But, what if we reduce the number of items and instead enforce multiple conditions as ranking instructions? It turns out, LLMs have a long way to go. See our new paper:
arxiv.org/abs/2404.00211. 1/n
Hayate Iso retweeted
The dataset proposed in our NAACL paper has been released! Let's analyze the capability of LLMs in terms of the fact-level popularity📈
github: github.com/megagonlabs/witqa
#NAACL
📢Accepted to #NAACL2024 main conference🎉
Want to evaluate the ability of LLMs from viewpoints of fact-centric analysis?🤔 We construct a new dataset, WiTQA, including questions with various popularity levels and present a comprehensive analysis.
📖: arxiv.org/abs/2402.13492
Hayate Iso retweeted
Hayate Iso retweeted
Frontier models all have at least 100k context length, Gemini 1.5 has even 1m context. What about research and open source?
Introducing Long Context Data Engineering, a data driven method achieving the first 128k context open source model matching GPT4-level Needle in a Haystack
Hayate Iso retweeted
📢 We invite authors of accepted EACL Findings papers, without allocated presentation slots, to present their work at the NLP4HR workshop at #EACL24.
Deadline: February 1st, 2024, 11:59 pm AoE
Find the application on our website: megagon.ai/nlp4hr-2024/
#MachineLearning #NLProc
Hayate Iso retweeted
Explore key challenges of applying #NLP to #HR tasks. Through the @NLP4HR Workshop, we explore challenges of skill identification, person-job fit, and bias mitigation, using insights from LMU, OECD, Indeed, and more! #EACL2024 #NLProc #AI #MachineLearning
megagon.ai/first-nlp4hr-work…
Hayate Iso retweeted
“Zero-shot Triplet Extraction by Template Infilling”, was accepted to Asia ACL 2023! Congratulations to Bosung Kim, Hayate @Iso_map, Nikita Bhutani, @EstevamHruschka, Ndapa Nakashole, and @TomMMitchell!
@aaclmeeting #AI #NLProc #MachineLearning #LLMs megagon.ai/publications/zero…
Hayate Iso retweeted
Join us as we chat with our talented summer interns. Get a glimpse of what our interns worked on and their experience at Megagon Labs. Read our Megagon Summer 2023 Internship Experience article:
megagon.ai/megagon-summer-20…






















