
Principal Engineer at @NVIDIA working on programming languages. @adspthepodcast co-host. C++ Library Evolution chair emeritus. Frequent flyer. Horology fan.
Manhattan, NY
Joined March 2011
- Tweets12.6K
- Following3.1K
- Followers21.2K
- Likes1.1K
Pinned Tweet
The latest revision of @INCITS/@isostandards COBOL comes out this year
The goals of COBOL sound normal today:
- Portable
- Freely available
- Designed by the community
In 1959 it was radical & unprecedented
It was also conceived of & led by women
This is the story of COBOL
This is my last week at @NVIDIA.
It's been an honor and a pleasure to be a part of this incredible team.
I couldn't have asked for a better job these last 9 years.
🫡
I joined @nvidia nine years ago today, against the advice of a colleague who said their comp was below market.
But I really believed in the mission and the people, so I went.
I met @__simt__, Michael Garland, and Jared Hoberock on the C++ committee. I thought they were the smartest people there.
I worked at @BerkeleyLab on concurrent and parallel programming paradigms, mostly for SIMD.
@__simt__ convinced me that SIMT was a superior model for parallelism.
He sold me on a revolutionary new GPU architecture where threads would independently scheduled: Volta, the basis of all modern GPUs.
I asked him three times to find me a place at @nvidia. The first two times, he wasn't sure where to place me or if I'd like it.
The third time, he had me come in to talk to Chris Lamb (now VP of CUDA). We chatted for two hours and he outlined the role he believed modern C++ would play in CUDA.
I came back for an 8 round interview panel. All of the questions were answered with std::atomic. No one was sure what I would be working on.
I, a modern C++ person, was placed in the CUDA driver org, which was written almost entirely in C.
On my first day, my manager told me I'd be responsible for Thrust and CUB, @nvidia's first two open source libraries. No one had worked on them for a while and there was no product manager. I'd have to do it all.
I put together a roadmap and got the libraries back on GitHub. I started sending status updates from CCCL, the CUDA Core Compute Libraries team, which did not exist. Eventually, they let me hire three people to fill out the team.
We built libcu++ and abstractions for all the quirky new features being added to our chips to support these AI workloads that seemed to be growing in importance rapidly.
After a few years I was asked switch orgs and guide the roadmap for our HPC C++ compiler. Programming languages and models quickly became my role, from stdpar to CUDA-Q to CUDA Python to CUDA Tile to autoresearch.
What a ride it's been!
It's been a pleasure working with you 🫡
Today I’m joining @Natl_Compute
I’ve spent the last four years working on GPU access, first as a founding engineer at Brev and then at NVIDIA after they acquired us in 2024. When I started, on-demand GPU capacity at the hyperscalers was effectively gone. If you were an independent researcher, a two-person startup, or a grad student with a grant and a deadline, you refreshed console pages and got nothing, or hammered the API only to get “insufficient capacity” errors back. I vividly remember hard-coding a filter at Brev to hide certain GCP instances because they almost never provisioned.
Around 2023, a wave of new public clouds were coming online, and many of them had on-demand capacity. That gap is a big part of why Brev worked: we gave researchers a simple way to find GPUs across dozens of providers and start building. For a while, aggregation was enough.
Then in 2025, I watched that capacity dry up too.
Access to compute has never been worse. Existing supply is already under contract, and new capacity is reserved before it ever comes online. I won’t rehash why. There are a hundred good explanations out there covering debt financing, anchor tenants, backstops, lead times and so on.
What matters more are the downstream effects. Long-term, single-tenant leases hold back scaling for everyone who couldn’t secure funding in time, and the industry’s momentum shouldn’t hinge on the handful of players who locked up access early.
Today small startups, labs, and university researchers don’t even get a chance to compete. National Compute is solving this with a foundational layer we call “the Grid”. It’s an interconnected network of multi-silicon capacity composed of multiple providers and spans multiple sites that will be pooled for our members.
The speed at which this team has been able to scale in such a small fraction of time is remarkable, and I’m excited to push that scale even higher for the public benefit.
Bryce, the CUDA Colonel retweeted
Today I’m joining @Natl_Compute
I’ve spent the last four years working on GPU access, first as a founding engineer at Brev and then at NVIDIA after they acquired us in 2024. When I started, on-demand GPU capacity at the hyperscalers was effectively gone. If you were an independent researcher, a two-person startup, or a grad student with a grant and a deadline, you refreshed console pages and got nothing, or hammered the API only to get “insufficient capacity” errors back. I vividly remember hard-coding a filter at Brev to hide certain GCP instances because they almost never provisioned.
Around 2023, a wave of new public clouds were coming online, and many of them had on-demand capacity. That gap is a big part of why Brev worked: we gave researchers a simple way to find GPUs across dozens of providers and start building. For a while, aggregation was enough.
Then in 2025, I watched that capacity dry up too.
Access to compute has never been worse. Existing supply is already under contract, and new capacity is reserved before it ever comes online. I won’t rehash why. There are a hundred good explanations out there covering debt financing, anchor tenants, backstops, lead times and so on.
What matters more are the downstream effects. Long-term, single-tenant leases hold back scaling for everyone who couldn’t secure funding in time, and the industry’s momentum shouldn’t hinge on the handful of players who locked up access early.
Today small startups, labs, and university researchers don’t even get a chance to compete. National Compute is solving this with a foundational layer we call “the Grid”. It’s an interconnected network of multi-silicon capacity composed of multiple providers and spans multiple sites that will be pooled for our members.
The speed at which this team has been able to scale in such a small fraction of time is remarkable, and I’m excited to push that scale even higher for the public benefit.
hundreds of MI355x and B300 nodes
now live on nationalcompute.com
subsidized for all .edu and .govs
it’s time to scale the frontier ecosystem
for the public benefit
Bryce, the CUDA Colonel retweeted
This keynote by Pat Hanrahan at #RacketCon is phenomenal! Einstein summation, Penrose diagrams, APL, J, K, Cajori's A History of Mathematical Notation, Bringhurst's The Elements of Typographic Style and more! 🤯 youtube.com/watch?v=pIoogidK…
Bryce, the CUDA Colonel retweeted
Pat mentioned that the keynote was a lecture from a DSL course he taught. I created a site that links to that course / other resources. Can't believe Pat was @ia_buck's advisor (created Brook, the predecessor to CUDA) and I had never heard of him codereport.github.io/vibing/…
This keynote by Pat Hanrahan at #RacketCon is phenomenal! Einstein summation, Penrose diagrams, APL, J, K, Cajori's A History of Mathematical Notation, Bringhurst's The Elements of Typographic Style and more! 🤯 youtube.com/watch?v=pIoogidK…
This has been a huge problem for me.
I've been skeptical of people doing auto research with a system that preserves knowledge/learnings/patterns from prior experiments.
How do you prevent bad learnings from getting into the knowledge base?
Bryce, the CUDA Colonel retweeted
I am looking for a summer 27 intern to join my GPU performance engineering team (Bay Area, CA) We are working on some pretty cool stuff and your code will directly contribute towards advanced semiconductor node manufacturing!
Application link: kla.wd1.myworkdayjobs.com/Se…
Bryce, the CUDA Colonel retweeted
Merged into @nvidia. Repo: TileGym NVIDIA/TileGym#211
Up to 7.2× less peak memory (61.5 GB → 8.5 GB at 16 pairs) and 1.13–1.27× faster than unfused PyTorch on a fused DPO loss, running on cuTile. Measured at T=1024, H=4096, V=128256, bf16, on an RTX PRO 6000 Blackwell.
It never materializes the full logits. The gradient is one scalar times softmax, onehot, which NVIDIA’s own tuned cross-entropy kernel already writes in place, so the whole op is one kernel launch plus the GEMMs. At 1 pair it holds slightly more memory than the unfused path; the savings come from 2 pairs up, and the chunked path takes over at 16.
The first version was ~1.5× slower than the baseline. Profiling showed an fp32 GEMM on CUDA cores eating over half the runtime. Fixing it led to my earlier merge, NVIDIA/TileGym#212 · [liger] fused_linear_cross_entro…: 2.4× faster, 3.2 GB less memory.
Thank you to @blelbach and Hannah + TileGym maintainers for the careful reviews and the fast turnarounds.
More to come. Next up: ORPO, SimPO and KTO on the same skeleton. I’ve just started.
Thank you for the incredible SDKs: @NVIDIAAI
Machine learning is older than we thought! Did you know the Confucians invented Dirichlet Latent allocation in the third century?
Bryce, the CUDA Colonel retweeted
Kimi K3 decode on NVIDIA GB300 NVL72: 842 tokens/s at batch 1 with DSpark 🚀
A unified CuTe DSL decode path fuses attention, MoE and communication, delivering 1.5–2× higher decode throughput in several tested configurations and up to 1.8× higher P90 interactivity on the @SemiAnalysis_ AgentX workload.
The post covers the kernel work, the correctness and long-context evals behind it, and benchmark sweeps across batch sizes, context lengths and parallelism configurations.
Full write-up: vllm.ai/blog/2026-10-16-kimi…
Great work and collaboration with the @NVIDIA team!
Bryce, the CUDA Colonel retweeted
📢 ADSP Episode 308 is out now! In this episode, @ben_deane and @code_report chat with Jason Turner @lefticus about his Rule of N talk from @CppCon and @NDC_Conferences TechTown 🎉 adspthepodcast.com/2026/10/1… – at Richmond Hill, Ontario
Bryce, the CUDA Colonel retweeted
In early 2022, the first “real” @modal user was using us for processing large scale weather data. We were anxious about going up to them and asking them to pay ~$500 for continuing to run these jobs. (We thought it was a lot!)
Obviously the numbers are much bigger now.
But we’ve also built so much more since then, and our velocity and ambition are growing even faster. This year, we figured out how to scale to millions of concurrent sandboxes, launch RDMA clusters securely and elastically for distributed training, train state of the art speculators for inference, build a whole new sandbox runtime, and more. So many great rabbit holes, and we’re going deeper down all of them.
If this sounds interesting to you, we’re hiring! modal.jobs
Today we're announcing that Modal has crossed $1B in annualized run rate, as well as raised a $750M Series D at a $15.75B valuation, led by @Accel.
Since our Series C, usage has tripled and we’ve grown more than 15x year-over-year.
We are deeply grateful to the developers and companies building on Modal. There's still so much more to build, and we can't think of a better time to be building software.
Today we're announcing that Modal has crossed $1B in annualized run rate, as well as raised a $750M Series D at a $15.75B valuation, led by @Accel.
Since our Series C, usage has tripled and we’ve grown more than 15x year-over-year.
We are deeply grateful to the developers and companies building on Modal. There's still so much more to build, and we can't think of a better time to be building software.



































