Moved to tsrs over the weekend. Drop-in replacement. Around 5x faster for OC. Kind of amazing that it took Microsoft a team and two years to rewrite the TS compiler in go, and now you can just let an agent rip and get sth significantly better within weeks.

Pi is one of the coolest projects in the AI world today. I'm on their team. Giving developers and companies choice of model, control over the runtime, data, skills, plugins… Excited to have great support on @vercel: vercel.com/blog/making-every…

FP4による大規模言語モデルの事前学習について、既存手法を体系的に比較した論文が公開されている。 “What Matters for NVFP4 Training - A Comparative Study of Low-Precision Pretraining Recipes” FP4を使った学習は高速な行列演算や必要なメモリ帯域幅を抑えられる一方、BF16と比較すると精度低下や学習の不安定性が残る。 FP4のE2M1形式では、0、0.5、1、1.5、2、3、4、6と、それらの負の値からなる15種類の数値しか表現できない。ただし、NVFP4では16個の値からなるブロックごとにスケール係数を持ち、それを掛けることで表現範囲を調整する。 今回の論文では、8BのDenseモデルと30B-A3BのMoEモデルを最大1Tトークン学習し、既存のFP4学習手法を比較している。 特に提案するMosaic NVFP4では、以下の工夫を組み合わせている。 ・Forwardでは重みを1次元で量子化し、Backwardでは必要な方向に再量子化する。 ・Backwardの2種類の行列積にRandom Hadamard Transformを適用し、外れ値の影響を軽減する。 ・勾配にはMS-EDEN（後述）を使い、Stochastic Roundingより小さな量子化誤差で不偏性を実現する。 ・その他のテンソルにはFour Over Sixによる適応的なスケール選択を適用し、量子化誤差を削減する。 ・最後の15%の層はBF16のまま維持する。 これらを組み合わせることで、30B-A3Bモデルでは1Tトークン学習後のBF16に対する相対学習損失差を0.81%未満まで縮小し、従来のFP4手法から0.36パーセンテージポイント改善した。 特に厄介なのは、すべての層をFP4にすると、8Bモデルでも300Bトークンを超えたところで学習が発散したことである。また、手法間の優劣は150Bトークンを超えるまで安定せず、短期間の小規模実験だけではFP4学習手法を適切に評価できないことも示された。 ・MS-EDEN 勾配の量子化において、バイアスをゼロに保ちながら量子化誤差の分散を小さくする手法。 個々の値を確率的に丸めるStochastic Roundingとは異なり、まずベクトル全体にRandom Hadamard Transformを適用して値を混合し、FP4に量子化する。その後、補正係数を掛けることで不偏性を回復する。 この補正係数はNVFP4のブロックスケールに組み込まれる。スケールはFP8で格納されるため、補正後のスケールを量子化する際にはStochastic Roundingを使用する。 これにより、各要素を確率的に丸める場合よりも小さな量子化誤差で、不偏な勾配推定を実現できる。 ・Four Over Six FP4のE2M1形式では、表現可能な値の4と6の間に5が存在しないため、この領域で量子化誤差が大きくなりやすい。 通常のスケーリングでは、ブロック内の絶対値の最大値をFP4の6に対応させる。しかし、最大値を4に設定した方が、ブロック全体の量子化誤差が小さくなる場合がある。 Four Over Sixは、ブロックごとに最大値を4に対応させる場合と6に対応させる場合の両方を試し、量子化後の二乗誤差が小さい方を選択する。 コメント ==== FP4は行列積の演算性能を向上させ、必要なメモリ帯域幅も削減できるため有望だが、大規模な事前学習での実績はまだ限られている。 今回の研究でも、かなり学習が進んだ後に発散する現象が確認された。最後の15%の層をBF16に維持することで発散を抑えられ、この条件ではBackwardの量子化手法による性能差もかなり小さくなる。 これは、Backwardの量子化バイアスを抑えるだけでは解決できない問題が残っていることを示している。著者らは量子化による表現精度の問題を示唆しているが、発散の詳しい機構はまだ分かっていない。 学習が進むにつれて小さな数値誤差が蓄積し、学習ダイナミクスが変化する可能性も考えられる。Forward側の表現誤差や、量子化とパラメータ更新の相互作用も重要だと思われる。 これはFP4に限らず、誤差逆伝播法を近似したり、一部を別の計算方式に置き換えたりする手法にも共通する課題である。小規模な実験では問題なく動作していても、モデルを大きくしたり学習を長期間続けたりすると、発散や性能劣化が生じる場合が多い。 深層学習はかなり粗い数値計算でも動作する印象があるが、特に長期間の学習における勾配計算や数値誤差の扱いは、想像以上に繊細である。

How do you _guarantee_ your agent's code is correct? You prove it. Introducing Proof Driven Development Kit (PDD Kit) from @MSFTResearch. PDD Kit helps turn your plan into a formal spec, then proves the agent's code meets that spec for every input. No vibe verification. PDD Kit is integrated with @GitHubNext's Chopin, so you can plan as a team. PDD Kit takes an app's login flow from intent to a verified pull request: It discusses your code, infers your intent as it goes, and proposes a formal spec in Verus, a proof language for Rust. Once you approve the spec, the Verus verifier automatically proves the code meets it. Where proof isn't possible, it falls back to empirical verification with tests and computer-use agents (CUAs). It works on Rust today. No new language, no change in how you produce the underlying code, making formal methods more accessible. The goal is a verified software factory, where engineers review specifications instead of code. Come see it in action at GitHub Universe, Oct 28–29. Swing by the Speakeasy and ask for GitHub Next!

Nemotron-3.5-Embed is now available in turbopuffer native embedding on our enterprise PDF search benchmark, Nemotron-3.5-Embed 8B improves recall@20 by ~5% over the next best model turbopuffer.com/docs/embeddi…

【ブログ】PLaMo Post-trainingチームの夏期インターンに参加した齋藤慶汰さんによる寄稿記事です。LLMが生成したベンチマークが妥当なベンチマークになっているかを検索agent向けベンチマークを題材に調査していただきました。 preferred.jp/ja/blog/tech/ll…

We're changing the game for multi-speaker separation in the real world, at scale. Introducing Luel Multi-Speaker Separation. One messy conversation in. Every voice back on its own track, at up to 48 kHz. Trained on real conversations from our contributor network of 900k+ users, not just studio speech.

Today Monaco is launching Agent Matrix. Monaco unleashes countless agents working together, goal seeking, with one outcome in mind: accelerate revenue growth. If you don’t have an army of agents working 24/7 to accelerate revenue growth, you’re living in the past. Apply for early access now.

We formally announced Mirendil less than 4 months ago. Today, I'm excited to share an important milestone for: Pretraining at scale without a pretraining team. Mirendil's mission is to solve Recursive Self-Improvement (RSI) and use it to democratize frontier AI development, ultimately accelerating science and technology. At Mirendil, when we think about RSI, we think holistically about the company as an intelligent entity that self-improves toward its goal. From that perspective, the most powerful notion of acceleration is reducing the time and resources (number of human experts, compute, and other costs) needed to achieve ambitious goals. Given that pretraining is perhaps the most important yet most challenging part of frontier AI development, we decided to run an ambitious controlled experiment: Starting fully from scratch, can we significantly reduce the time, number of human experts, and compute resources needed to go all the way to successfully pretraining a strong coding model at scale? Mirendil-Base is the result of this experiment. Our first pretrained model was built by one researcher in 3.5 months, with less than $10M in compute. This includes all the research and small-scale experiments it took to get there, as well as the final training run. The recipe holds up as the models get bigger. For reference, Mirendil-Base is in the 100B-500B parameter range, Mirendil-Base-Large in the 500B-2T range, and Mirendil-Base-XL in the 2T-10T range. We're beginning to see a world where a single researcher, empowered by AI, can take on work that once required an entire research organization. As these systems improve, the bottleneck shifts from the size of a team to the ambition and judgment of the people directing them. This has profound implications for who gets to build AI and, ultimately, who gets to push the frontiers of science and technology forward. We’ll share independent evaluations, post-training benchmarks and API-access sooner than you think so stay tuned!

Jensen closed his GTC Berlin 2026 keynote with this Hermes Agent demo as the finale. He introduced it by saying the way to build an agent is to ask an agent to build it for you, so he installed and set up a customized Hermes in one prompt. Then he used Hermes, running on @nvidia RTX Spark, to create a simulated training environment for the @huggingface Microduck. Hermes planned an obstacle course with Nemotron and had GPT-6 Astra build it using FreeCAD, Omniverse libraries, Blender, Newton, and Isaac Lab. Hermes trained thousands of simulated ducks on that course until the validated policy was ready to run on the real robot in the physical world. At Nous, we believe open source is a practical tool as much as a philosophical ideal. It gives users the ability to build powerful, complex workflows chaining numerous local+cloud models, devices, and tools themselves instead of waiting for a company to enable them.

Say hi to qev-0.8b the tiny local decision model. 3.5 ms a decision · 0.8B · 1.3 GB · Apache-2.0 #1 in its weight class (77.1% on Benchmark Heaven, vs 26 others) Text, images, games. Runs on your device.

Build specialized harnesses, folks. This is a banger paper from Google Cloud AI Research on self-evolving agent harnesses. (bookmark it) They introduce OO-Harness, which raises AutomationBench from 61.6% to 77.4% with the same Gemini 3.5 Flash executor. It also beats Meta-Harness on all three benchmarks they test. More details below. Instead of evolving one global harness for every task, an evolution agent builds a class hierarchy of specialized harnesses from execution records. The design has three parts. > Subclasses inherit the base harness and override only the methods a task cluster needs > Design lessons record which changes helped and which ones failed > A router reads the task and the base run, then picks a subclass or keeps the base result On Workspace-Bench, the score goes from 25.5% to 39.5%, and on Toolathlon-GYM strict pass goes from 31.2% to 37.5%. Spending extra inference on best-of-K sampling or self-refinement does not reliably help. Inheritance also improves results. On Workspace-Bench, harnesses built as subclasses score 4.96 points higher than the same evolved code copied into flat classes. arxiv.org/abs/2610.28380 Chat with Paper: academy.dair.ai/papers/objec…

API version 2026-10-21 is now available. The Claude API has evolved a lot, but this is our first official version release since 2023. We plan to release versions more often so you can handle breaking changes before a new model requires them. 2023-06-01 is not deprecated.

HeyGen Voice Turbo takes the #1 spot on the Artificial Analysis Controlled Voice TTS Arena Leaderboard, with their previous model, HeyGen Voice, ranking #2 Just two weeks after reaching #1 with its previous model, @HeyGen has released HeyGen Voice Turbo, which now leads the leaderboard at $4/1M characters. Our Controlled Voice Arena evaluates models using the same 8 cloned voices (4 US, 4 UK), allowing listeners to compare speech quality across models independently of voice preference. Key takeaways: ➤ Controlled Voice Arena: HeyGen Voice Turbo has an Elo of 1,202 across 1,663 appearances, ranking #1 of 46 models, ahead of HeyGen Voice at 1,196 and Qwen-Audio-3.1-TTS-Plus at 1,185. It also ranks #1 in Assistants and Entertainment. ➤ Accents: HeyGen Voice Turbo ranks #1 in English (UK) with an Elo of 1,197 and #2 in English (US) at 1,204, behind only Qwen-Audio-3.1-TTS-Plus. ➤ Pronunciation Robustness: HeyGen Voice Turbo scores 87.8%, ranking #8 of 33 models, improving from HeyGen Voice's 83.2%. It also ranks #1 in Preserving Exact Sequences at 91.8%. ➤ Pricing: At $4/1M characters, HeyGen Voice Turbo is the lowest-priced model on the Controlled Voice Arena Leaderboard, compared to Simba 3.2 at $6.6/1M characters and Gemini 3.8 Flash-Lite TTS at $11/1M characters. It is also 7.5x cheaper than HeyGen Voice at $30/1M characters. See more details and listen to samples below ⬇️

My agents sent me 355 hours of reading this week. I can't read that fast. We're launching Spacesheep (YC F26) to fix that: agents' work lands in a space your team can actually understand and work on together. Try it: tell any agent "share this to spacesheep.dev"