Introducing Inference Router: enterprise-grade inference on demand. Kimi K3, GLM 5.3, DeepSeek V4.1 Flash, Qwen3.8 Flash, Muse Spark 1.3, Mercury 3 and more, available with an uptime SLA of at least 99.9%. US hosting and web search built in. you.com/router

we're doing a massive update on Wafer's popular open source guide to GPU performance engineering! a few cool resources going into this update: ⠀ 1. Stream-K: why a matmul can leave part of the GPU idle while the last blocks finish, and how dividing the multiply-accumulate work more evenly helps. ⠀ 2. tcgen05 for dummies: gau-nernst builds a Blackwell matmul step by step. Tensor Memory, asynchronous copies and pipelining, with code you can follow as the kernel develops. ⠀ 3. Thinking Machines' Defeating Nondeterminism: how changing the batch can change the order of floating-point additions, and why that can change inference results. then, how to make the kernels batch-invariant. ⠀ 4. Triton's Linear Layouts: how tensor elements are arranged across threads and registers, and the data movement needed to change that arrangement. ⠀ the update also expands the quantization, speculative decoding and hardware references. ⠀ the guide starts with one inference request, then moves through GPU fundamentals, kernels, serving engines and distributed systems. start at the beginning if you're new; use the individual sections when you hit a specific problem. ⠀ if you know a primary source that explains one of these topics better, open a PR.

we launched the most comprehensive ai performance engineering repo in the world we're posting every single resource follow and save to keep up with the series. links in thread 🧵 part 14: Blackwell Tuning Guide NVIDIA covers the execution limits, memory hierarchy, and communication mechanisms behind tuning AI workloads on Blackwell: - register budgets, resident warps, and occupancy in fused AI kernels. - shared-memory limits for attention and matmul tiles across Blackwell compute capabilities. - thread-block clusters and distributed shared memory for exchanging tensor data across blocks. - 8-block portable clusters, B200's optional 16-block clusters, and the parallelism tradeoff of larger clusters. - L1/shared-memory carveouts and the balance between caching and explicit tensor staging. - L2 cache persistence for inference kernels that reuse the same tensors. - NVLink peer access and direct tensor transfers between GPUs in model-parallel inference. the guide connects Blackwell's hardware limits to kernel design decisions like tile size, block cooperation, and tensor reuse, helping engineers choose what to test when tuning attention, matmul, and multi-GPU inference.