@raydistributed
Account based inUnited States
About this account
- Account based in
- United States
- Connected via
- Web
Account-level information from X, not a live location or the device used for a specific post.
A distributed compute framework for scaling AI workloads. Created and developed by @anyscalecompute.
Joined August 2019
- Tweets2K
- Following2
- Followers11.7K
- Likes2K
ray retweeted
Learn how to productionize large language models using distributed post-training and inference optimization techniques. Participants will explore advanced LLM post-training techniques, including agentic reinforcement learning with SkyRL.
Hands-on topics include:
- Intro to LLM post-training
- Intro to RLVR (with GRPO)
- Recent development of LLM post-training:
- On-policy distillation (OPD) and Multi-teacher On-policy distillation (MOPD)
- Case study: Post-training a terminal agent
- Building evals
- Building custom environments
- RL systems
- Demo walkthrough
- Intro to SkyRL
- Full Async RL with SkyRL
- Performance monitoring and tuning for RL rollouts
Register: na2.hubs.ly/H08wNCZ0
ray retweeted
If you’re interested in the low-level algorithmic and systems details of how to scale training to 1M-token context, read this post from our team.
Super excited to share some of the work I’ve been doing on supporting 1M token context training on SkyRL for GLM 5.3 Flash, 64x higher than the out of the box Megatron baseline! 🚀
This work was done in collaboration with the amazing folks at @LilaSciences! 🤩
Getting there took key algorithmic and systems optimizations in four main areas:
1) DeepSeek Sparse Attention: run DSA on a sparse kernel; chunk and shard its quadratic indexer.
2) Activation recompute and offloading: recompute activations in backward and offload saved inputs to CPU.
3) Context parallelism: split each sequence across GPUs, with new CP support for KDA and a fix for DSA.
4) Avoiding unnecessary precision upcasts: keep FP32 copies out of memory and off the network.
For more details on our implementations and our results, check out our full technical blog here: blog.skyrl.ai/blog/1m-contex…!
This work benefited from the contributions of many people along the way. Big thanks to @pcmoritz, @erictang000, @sumanthrh, @CyrusHakha, @richliaw from @anyscalecompute and Keith Truongcao, Tushar Khot, Vivek Subramanian, Semih Kara from Lila Sciences for all the help and guidance!
ray retweeted
When one region runs short on GPU capacity, @ChefRobotics jobs on Anyscale try others automatically until one comes through, instead of risking days without H100 access.
See how the team scales robot training: na2.hubs.ly/H08qgcl0
ray retweeted
Super excited to share some of the work I’ve been doing on supporting 1M token context training on SkyRL for GLM 5.3 Flash, 64x higher than the out of the box Megatron baseline! 🚀
This work was done in collaboration with the amazing folks at @LilaSciences! 🤩
Getting there took key algorithmic and systems optimizations in four main areas:
1) DeepSeek Sparse Attention: run DSA on a sparse kernel; chunk and shard its quadratic indexer.
2) Activation recompute and offloading: recompute activations in backward and offload saved inputs to CPU.
3) Context parallelism: split each sequence across GPUs, with new CP support for KDA and a fix for DSA.
4) Avoiding unnecessary precision upcasts: keep FP32 copies out of memory and off the network.
For more details on our implementations and our results, check out our full technical blog here: blog.skyrl.ai/blog/1m-contex…!
This work benefited from the contributions of many people along the way. Big thanks to @pcmoritz, @erictang000, @sumanthrh, @CyrusHakha, @richliaw from @anyscalecompute and Keith Truongcao, Tushar Khot, Vivek Subramanian, Semih Kara from Lila Sciences for all the help and guidance!
ray retweeted
@StimulerApp runs 4 AI models simultaneously for every English coaching session.
Speech understanding. Assessment. Proficiency tracking. Speech generation.
Each fires at a different rate. Each needs its own compute. One three-person ML team runs it all.
900K+ speeches a day on Ray and Anyscale
Case study: na2.hubs.ly/H08qhcX0
ray retweeted
In this session, @Marwan1112, AI Engineer at Anyscale, will walk through:
- Building Ray Serve applications with deployments, composition, and load-aware request routing
- Deploying and scaling LLM endpoints with Ray Serve LLM on engines like @vllm_project and @sgl_project
- Optimizing the inference engine with the KV cache, paged attention, prefix caching, and continuous batching
- Scaling frontier models with KV-cache-aware routing, prefill/decode disaggregation, and wide expert parallelism for mixture-of-experts (MoE) models
- Deploying Custom LLMs on Anyscale and Integrating Them with Agentic Platforms (Claude Code, Cursor, Codex)
- Comparing costs across self-hosted GPUs, subscriptions, and token-based API billing.
Register: na2.hubs.ly/H08hflG0
ray retweeted
Kubernetes doesn’t know what a Ray cluster is. So teams keep hand writing the same Deployments, Services, and recovery scripts to run Ray on it.
KubeRay fixes that: autoscaling, fault tolerance, and zero downtime upgrades for Ray, built into Kubernetes.
na2.hubs.ly/H08qhRS0
ray retweeted
@Xoople processes 500,000 km² of satellite imagery in under 5 minutes on 12 A10 GPUs with Anyscale on @Azure.
A single request can fan out to 2 trillion pixels. 15 engineers without a dedicated infra team.
Read the case study: na2.hubs.ly/H08qgTD0.
ray retweeted
Run Reflection's Beam at scale in your own account for full sovereignty but without managing infra with Ray Serve LLM on Anyscale!
Excited to announce Day 0 support for Reflection's Beam model on Ray Serve LLM!
Beam is Reflection's first open-weight model, advancing the frontier of Western AI in the 500B-parameter class. Built as a sparse MoE model, Beam is optimized for coding, agentic workloads, and efficient inference.
We've validated Ray Serve LLM deployment recipes for Beam along with its DFlash2 drafter. 🧵
ray retweeted
Open Source AI Week kicks off tomorrow 🎉
📍 The Midway, SF, 6pm 👇
na2.hubs.ly/H08nMYB0
Join us alongside other key players in AI helping organizations adopt and scale their use of open-source.
ray retweeted
We post-train GLM-5.3 Flash in a sandbox that reproduces the Bits Investigation tools over archived scenario data.
Our async RL stack is based on #SkyRL, with efficiency and reliability enhancements in our internal fork. Training and inference run concurrently, with a staleness budget for off-policy updates.
ray retweeted
datadog ai research used SkyRL to post-train GLM 5.3 Flash to perform at the pareto frontier for their SRE eval
@bthecohen and team were some of the earliest users of SkyRL at scale, really cool to see these results finally out! great tech report as well
Meet Scout 1.0, our first AI model built for investigating production incidents.
Scout matches frontier-model investigation quality at a fraction of the cost, which makes it practical to investigate more issues, earlier.
We trained Scout on realistic simulations built from real-world incidents and production telemetry, using reinforcement learning to reward accurate root causes backed by evidence.
Learn more and sign up for the preview: bit.ly/3TJknGO
ray retweeted
Great to see this out!
Datadog's new SOTA model for investigating production incidents. Their in-house training stack builds on top of SkyRL! 🚀🚀🚀
Meet Scout 1.0, our first AI model built for investigating production incidents.
Scout matches frontier-model investigation quality at a fraction of the cost, which makes it practical to investigate more issues, earlier.
We trained Scout on realistic simulations built from real-world incidents and production telemetry, using reinforcement learning to reward accurate root causes backed by evidence.
Learn more and sign up for the preview: bit.ly/3TJknGO
ray retweeted
Datadog post-trained GLM-5.3 Flash via RL on real-incident sims. Matches frontier SRE performance at much lower cost.
Their RL stack is built on top of SkyRL! Check out the technical report web-assets.dd-static.net/425…
Meet Scout 1.0, our first AI model built for investigating production incidents.
Scout matches frontier-model investigation quality at a fraction of the cost, which makes it practical to investigate more issues, earlier.
We trained Scout on realistic simulations built from real-world incidents and production telemetry, using reinforcement learning to reward accurate root causes backed by evidence.
Learn more and sign up for the preview: bit.ly/3TJknGO
ray retweeted
@ChefRobotics trains the Vision-Language-Action models behind its food assembly robots 10x faster on Anyscale, with 92% scaling efficiency across multi GPU runs on 16 A100s.
Read the story: na2.hubs.ly/H08q5bh0
ray retweeted
@Coinbase is building an agent-first ML platform where AI agents watch model performance and trigger retraining on their own.
Aman Choudhary on how Ray on Anyscale helps them run up to 1 billion inferences a day
na2.hubs.ly/H08nMPc0
ray retweeted
We’re joining Open Together with @huggingface and friends this Friday to support everyone building AI with open-source
SF, Oct 16, 6pm 👇
na2.hubs.ly/H08m4Mv0
ray retweeted
Our upcoming webinar will cover how to design scalable distributed inference systems for production AI applications, from classic ML models to large language models.
Learn how Ray Serve delivers high-throughput, low-latency serving, and how to evaluate the performance tradeoffs in autoscaling, batching, routing, and inference-engine optimization.
Topics include:
- Building Ray Serve applications with deployments, composition, and load-aware request routing
- Deploying and scaling LLM endpoints with Ray Serve LLM on engines like @vllm_project and @sgl_project
- Optimizing the inference engine with the KV cache, paged attention, prefix caching, and continuous batching
- Scaling frontier models with KV-cache-aware routing, prefill/decode disaggregation, and wide expert parallelism for mixture-of-experts (MoE) models
- Deploying Custom LLMs on Anyscale and Integrating Them with Agentic Platforms (Claude Code, Cursor, Codex)
- Comparing costs across self-hosted GPUs, subscriptions, and token-based API billing.
Register: na2.hubs.ly/H08hfbl0
