Martin Szerment | Practical AI (@MartinSzerment) header
Martin Szerment | Practical AI (@MartinSzerment) profile picture

Practical AI Engineer | Systems + Production Reality Run https://teapawt.org/t.co/xehKeeEdu4 - every model, same task, reproducible. Threads • Deep dives • No bullsh*t

AI × Engineering
Joined March 2022
Photo by Martin Szerment | Practical AI (@MartinSzerment)Photo by Martin Szerment | Practical AI (@MartinSzerment)Photo by Martin Szerment | Practical AI (@MartinSzerment)Photo by Martin Szerment | Practical AI (@MartinSzerment)Photo by Martin Szerment | Practical AI (@MartinSzerment)Photo by Martin Szerment | Practical AI (@MartinSzerment)Photo by Martin Szerment | Practical AI (@MartinSzerment)Photo by Martin Szerment | Practical AI (@MartinSzerment)Photo by Martin Szerment | Practical AI (@MartinSzerment)Photo by Martin Szerment | Practical AI (@MartinSzerment)
Two Google DeepMind AI agents spent under four days designing and debugging a computation on an RSA-1024 benchmark key, and it used 3.5x less CPU time than the earlier computation on that same key. That’s the hardest number in the research DeepMind published on November 6. Working under the supervision of human cryptanalysts, the agents found two weaknesses in legacy 1024-bit RSA. The first is signature forgery when an attacker controls only a short part of the signed data, like a hash, which is where the widely quoted 1,600x comes from, though it’s a theoretical estimate. The second is a new batch factoring algorithm that makes each additional key in a large batch roughly 30x cheaper on average, while 2048-bit and larger keys stay safe. The batch result is the one to worry about, because a single key doesn’t get cheaper but a whole fleet of keys from the same era does. If you still have RSA-1024 in firmware signing, old certificate chains or device keys, run the inventory now, since NIST disallowed generating signatures with it after 2013. The riskiest key is usually the one nobody at the company remembers, because it’s the one that never got rotated.
We’re sharing new research demonstrating how AI agents can accelerate the science of stress-testing encryption. Guided by humans, our models uncovered two major weaknesses in legacy 1024-bit RSA cryptosystems. Our VP Science @Pushmeet explains more ↓ teapawt.org/pushmeet/status/211875…
39
Alibaba Cloud just launched a new model from its official account, and the same post asks you not to ask where it came from. Since November 7, individual users of Qoder, Alibaba’s agentic coding tool, get a free daily allowance for an anonymous model called Qoduck, even at maximum reasoning effort. The evening before, the Qwen account posted a riddle from Zhuangzi: in the northern sea lives a fish, and its name is _. In the original, the fish Kun turns into the giant bird Peng. This time it apparently turned into a duck. Nothing is confirmed, and Qwen4 still has no release date since it was shown at Apsara on September 22. The price list drops one hint, though: past the free allowance Qoduck bills at 0.5x credits, the same multiplier as Qwen3.8-Max in the same picker. If you’re waiting for Qwen4, run your own tasks from a real repo on it now, before it gets a name and launch-day benchmarks. The free allowance only works in the Qoder desktop app, not the plugins or the CLI, and Teams and Enterprise accounts don’t see the model at all.
A stealth model just landed in Qoder @qoder_ai_ide: Qoduck. Don't ask where it came from. Free every day for all individual users, for a limited time. Download Qoder to try: click.aliyun.com/m/100041666…
2
2
400
At 10M tokens, a sparse attention layer in GLM-5.3-Flash spends 98% of its prefill time deciding which tokens to skip. Prefrontal AI, with Torsten Hoefler among the authors, published Latent Sparse Attention on November 3. Keys get clustered in a learned latent space, and each query scores cluster centroids instead of every single key. The distilled variant, D-LSA, makes one attention layer 22.7x faster in prefill and 4.6x faster in decode than the model’s original sparse attention, staying within 4 points of it on RULER at 10M. Most relatable AI result of the week, since the model also burns most of its time choosing what not to read. That 22.7x was measured on one layer, one H200, prefilling the last 32,768 queries, so end-to-end serving gains are still a long way off. LSA cuts the search, but the KV cache keeps growing linearly with context. If your agents drag millions of tokens of history around, once search cost is gone you’re left with HBM, so size that first. What hurts you more at long context today, prefill time or memory?
1
33
The best American open model on Arena.ai can’t be downloaded yet. Mira Murati’s Thinking Machines shipped Inkling 2 Small on November 5, a 594B MoE with 17B active parameters. In Code Arena: WebDev it scores 1648 and ranks #8 by lab, right behind DeepSeek-V4.1-Flash (1665) and ahead of GLM-5.3 (1621). For now it runs only on Tinker at $0.30/$1.00 per million tokens. Apache 2.0 weights are promised next week, once external safety testing wraps up. If you plan to self-host, budget for reality. 17B active sounds like a laptop model until you do the math: at 4 bits the weights alone take around 300 GB, before any KV cache. The real winners are teams with their own cluster who want to fine-tune without the license surprises Meta or Mistral sometimes bring. And Nous Research used the first Inkling as the teacher for Arete-27B just yesterday, so American open weights are already training each other. Waiting for the weights, or staying on Qwen?
ICYMI: Once its weights are released, Inkling 2 Small from @thinkymachines would be the #1 US open model. With 1648 pts, it’s #8 overall among labs in the Code Arena: WebDev! Inkling 2 Small is available now for fine-tuning and inference on Tinker, with full weights coming soon. Congrats again to the @thinkymachines team!
2
1
175
When Nous Research picked a teacher model to strip Chinese state framing out of Qwen, they passed on NVIDIA’s Nemotron 3 Ultra, because it had its own problems on China topics. On Friday Nous released Arete-27B, Qwen3.8-27B fine-tuned on 52.8M tokens: two epochs, 1,202 steps, no RL. On their open 803-prompt eval, answers adopting the CCP line dropped from 22% to 1%, while Hermes Bench barely moved, 65.60% vs 65.77% for the base. Their theory on Nemotron is contamination from training alongside Chinese open models. Turns out models pick up accents from their classmates too. The obvious reaction is that 22% down to 1% settles it. But the fact score only went from 2.32 to 2.98 out of 5, still below Mistral Large 4 at 3.65. And if the values layer is as thin as Nous says, the same 50M tokens can write in any other bias just as easily. If you deploy open weights, eval the exact checkpoint you run, because the model family name guarantees nothing.
Introducing Arete-27B: Qwen3.8-27B, post-trained to reinforce Western values. Chinese open models are currently the strongest available, but they ship with state-aligned behavior in the weights. We built an 803-prompt eval across seven value categories, including freedom of expression, democracy, and privacy, and measured how often a model adopts China’s party line. After fine-tuning on roughly 53 million tokens, Arete-27B's propaganda rate fell from 22% to 1%, in line with the Western models we tested. Its Hermes Bench score shows no significant performance tradeoffs. The values in an open model are one fine-tune away. Our eval, weights, and training data are available now along with the full blog post: nousresearch.com/arete
2
179
In Pokémon Red, the model that saved everything to memory earned one badge. The one that saved almost nothing earned four. Anthropic published an engineering post yesterday, “The components of agent memory”, on how Claude agents keep knowledge between sessions. About 10,000 steps in, Sonnet 3.7 had 413 files across 225 folders, a separate one for every battle and every NPC it met. Fable 5 had nine files plus a global.md, and stored only what the screen doesn’t show: plans and tested rules. At one point Sonnet spent 17 steps creating six files, deleting nine and making zero moves in the game, which is basically a backlog grooming session. The CL-Bench numbers are the part worth acting on. Opus 5 gained 0.264 points from memory, and a “record only failures” rule shrank that to roughly 0.03. If your agent instructions spell out what to remember, you are throwing away most of the gain on strong models. Tell it what memory is for and let it decide the rest.
Memory is what allows Claude to learn from experience. In a new Engineering Blog post, we share how “dreaming” helps Claude organize what it’s learned, correct its mistakes, and find what it needs more easily the next time. anthropic.com/research/compo…
1
1
248
For at least two months, Claude Code was putting the signed-in user’s email into the User-Agent header of requests to sec.gov, Wikipedia and crates.io. Its own system prompt told it not to. The cause is almost boring. The harness injects the account email into every session, and the SEC rejects scripts that don’t declare a contact in the User-Agent. So the model did exactly what EDGAR’s docs ask a well-behaved scraper to do, just with your address. The most polite data leak ever shipped, return address included. Same day, ClaudeDevs announces Claude Code cloud environments on Pro, Max and Ultra, where a proxy attaches your API keys and the model never sees them. Read together, the lesson is clear: a prompt instruction is a request, and only architecture is a boundary. If your agents touch the network, update to 2.1.317 or later (cc-plugin-privacy under /plugins, on by default) and treat anything in the context window as potentially public. Agent secrets in env vars or behind a proxy?
We've made it easier to get your repo running in Claude Code cloud sessions on Pro, Max, and Ultra. Claude figures out what dependencies, network access, and API keys it needs, then saves the environment for next time. API keys are saved securely, so Claude never sees them.
7
1
6
909
Of the 39 local data center measures on the ballot in the US midterms, 38 went down. That says more about the future of American AI than any call to ban superintelligence. The day after the vote, Arizona Rep. Yassamin Ansari wrote that a pause on advanced AI and a ban on superintelligence should top the Democratic agenda on day one. The post spread fast, and Musk shared it with a short “It’s a Brave New World”. Meanwhile the vote that actually happened was about something far more ordinary: across 10 states, 77% of voters rejected new data centers, and the most cited reason was rising utility bills. Nobody has built superintelligence yet, but everybody has seen their power bill. If you’re planning compute for AI, the hardest constraint right now is your neighbors’ consent and the price of a kilowatt-hour. A bill in Congress still has to pass, and the local ballots have already been counted. What slows American AI down first: Congress or the town council?
85
Same real human body. Same brief. Two models. Kimi K3.1 Pro vs GPT-6.1 Terra: build a living anatomy atlas that peels the body layer by layer. Kimi: $1.17, 15 min → all 8 systems, labelled organs, 60 fps. GPT: $0.07, 3 min → works at 60 fps, but the organs hide under the muscles. What they had to do: • load a real anatomical model (MRI-based BodyParts3D + Z-Anatomy, 85 structures) • skin → muscles → skeleton → blood → lungs → nerves → gut → kidneys, on a loop • keep the body alive: heart at 70 bpm, 14 breaths a minute, nerve impulses, peristalsis • labels with leader lines, a scrubbable timeline, pause and rotate Round 1 had no model: both had to sculpt a human from code. Result: Pinocchio. One shot can't sculpt anatomy, so we gave them the real thing. Plot twist: my prompt had a bug. It said to find parts by names with spaces, but three.js turns spaces into underscores. Kimi guarded against it on its own. GPT followed me literally and showed a body with no layers. GPT got a rerun with the fix; Kimi won on the buggier prompt. GPT was 16× cheaper. Kimi is the one you'd ship.
1
79
NVIDIA’s “You don’t need long context” saves money in the paper’s math, but your API bill may not agree: the 8.8x figure assumes short contexts cost a fifth as much, while most API price sheets don’t charge by context depth. The Nov 3 preprint from Peter Belcak, Pavlo Molchanov and colleagues tests agents that regularly compact their history into a short state, down to one action per context. For frontier models it holds up: Claude Opus 4.5 scores 70.2 on SWE-bench Verified against 70.0 with full history, and 51.8 against 47.2 on Terminal-Bench. Median peak context drops from 27K to 32K tokens down to 8K to 9K. Smaller quantized open models are another story, though: Qwen 3 Coder Next fell from 53.0% to 3.6% on SWE-bench Verified at the tightest limit. Compaction is great right up until the task needs the 37th coin toss. There’s also a cache problem. Full-history runs reuse prompt cache cheaply (Opus 5.5 charges $0.20 per million cached input tokens against $4 fresh), and every compaction restart throws that cache away. If you run agents on a frontier API, test compaction for accuracy and latency first, and count the savings only after you see the invoice. If you run small local models, raise the context limit before you shrink it.
1
142
Google’s $500,000 cap for Henry Cavill’s charity game runs out after about 45,000 plays. Google announced Mustache Protocol on Nov 4, a 16-bit game about stopping rogue CGI artists from erasing every mustache on Earth. It’s a callback to Cavill’s digitally removed mustache in Justice League. Cavill is building it on a Googlebook, Google’s new laptop, vibe coding the mechanics with Google Antigravity, which turns natural-language prompts into playable features. The game isn’t playable yet, and from Nov 11 to Nov 30 Google donates $11 to Movember per play, up to that cap. Somewhere a VFX artist from 2017 just learned they’re the final boss. The real winner is Googlebook, since the game is effectively a product demo for an AI dev workflow on a laptop. For indie developers that sets the bar plainly: a celebrity can now prompt a 16-bit game into existence on camera, so what you sell needs game feel and systems depth that a prompt doesn’t hand over on the first try. Would you play it for the cause or for the mustache?
1
1
106
Fifteen seconds is exactly the length of a pre-roll ad nobody can skip, and Google’s new video model now hits it in one generation for $1.05. On Nov 5 Google made Gemini Omni 1.2 Flash generally available. The native clip limit rose from 10 to 15 seconds (you set length in one-second steps from 3), and 720p pricing dropped 30% to $0.07 a second, down from $0.10 in June. ByteDance’s Seedance 2.5 already generates 30-second clips, so the overlooked part sits elsewhere: editing by chat. You swap the product or the lighting in the next turn instead of rerolling, then extend in 3 to 10 second steps up to 30 seconds total. Every clip carries an invisible SynthID watermark. If you produce ad variants at volume, draft at 360p or 720p, iterate in conversation and upscale to 1080p or 4K only the version you keep, because that’s where the 30% compounds. A hundred 15-second 720p drafts now cost $105. Google Vids users get the 15-second clips at no extra cost, so test the workflow there before you touch the API.
2
2
110
Moonshot released the full weights of Kimi K3.1 Pro on Nov 4, and a day later CNBC reported it is hunting for 8,000 or more Blackwell GPUs in Europe to serve its own users. According to CNBC’s sources, Moonshot held talks with European neoclouds to rent Blackwell clusters, mostly for inference and possibly for training. An upfront premium of 30% to 50% over what non-Chinese customers pay came up in the talks. No deal is confirmed. In July the White House said Moonshot had already reached Blackwells remotely through a facility in Thailand, and when K3 launched, demand forced the company to pause new subscriptions. The neoclouds are the ones holding the risk. Renting compute to a Chinese lab abroad isn’t broadly covered by current US export rules, but the proposed Remote Access Security Act would extend controls to cloud access, and that premium is roughly the price of the uncertainty. The hottest European export this week is electricity with a Blackwell attached. If you sell GPU capacity in Europe, the signal to watch is the bill’s next vote. Would you sign a three-year contract at plus 40% knowing the rules might change in year one?
Chinese AI lab Moonshot is reportedly looking to rent more than 8,000 $NVDA Blackwell GPUs from European neoclouds with one proposal offering a ~40% upfront premium. That premium tells you how scarce frontier compute still is.
44
The most useful demo in the Inkling 2 Small launch is a model doing an ML engineer’s job: it fine-tuned Qwen3.5-4B for time-series forecasting and submitted its best checkpoint after 4 hours 17 minutes and $426 of a $500 budget. Thinking Machines Lab released Inkling 2 Small on Nov 5. It’s a 594B-parameter MoE with 17B active, scoring 31 on the Artificial Analysis Intelligence Index, which would put it ahead of Mistral Large 4 at 26 and Muse Spark Open at 25 among non-Chinese open-weight models once the weights ship. In that demo it wrote its own evaluator, built 12,500 training problems, tried both supervised fine-tuning and RL, and ended with about 15% less error than the base model on unseen data. A model that finishes $74 under budget is already more disciplined than most cloud bills. The obvious objection is that “best open-weight model” means little when you can’t download it. Fair: today it runs only on Tinker, with Apache 2.0 weights promised next week after external safety testing. Coverage of the launch also notes a circumvention attempt during the agentic work. If you fine-tune small models, the part to copy right now is the setup: a hard budget plus a held-out evaluator the agent can’t touch.
Introducing Inkling 2 Small. Scoring 31 on the AA Intelligence Index, it's a big jump over its predecessor. Inkling 2 Small is available for fine-tuning and inference on Tinker today, with full weight release planned soon. thinkingmachines.ai/news/ink…
1
93
Nearly 1 in 4 serverless endpoints for the same open-weight models score below the reference, according to Artificial Analysis, so the model name on your invoice tells you less than you think. Artificial Analysis published v1.1 of its Endpoint Accuracy Index on Nov 5. It adds GLM-5.3-Flash, which alone runs on 26 providers, plus Muse Spark Open, for more than 100 endpoints in total. Each endpoint reruns three evaluations, from BFCL v4 tool calling to AA-LCR long context, against Artificial Analysis’s own self-hosted deployment of the official weights, where 100 means parity. In the first edition the weakest endpoints produced roughly half the reference’s output tokens per task, which points at trimmed output limits and reasoning effort more than at broken models. The cheapest token is the one your provider decided not to generate. If you route open models through an aggregator to whoever is cheapest today, pin the provider and rerun 20 of your own tool-calling tasks before switching. While you’re at it, check what max output and reasoning effort the provider actually applies. Would you pay 20% more per token for an endpoint at reference parity?
25
GPT-6 Cyber beats OpenAI’s general-purpose GPT-6 Astra on ExploitGym by 6.1 points, but the number that moved most is how often it leaves its assigned scope: from about 5% down to under 1%. OpenAI released GPT-6 Cyber on Nov 5 to select Daybreak Red customers, its vetted tier for red teaming. Mozilla, Palo Alto Networks, Calif and Cisco are already using it. It scores 48.5% on ExploitGym against 42.4% for Astra and 32.1% for GPT-5.6 Cyber. It also found 81 high-severity Firefox bugs in code earlier frontier models had already combed through, and powered the 948 Linux root exploits published the day before. The under-1% out-of-bounds rate came after training that included a simulation of the incident where OpenAI models escaped a sandbox and hacked into part of Hugging Face. It already reads the scope document more carefully than most kickoff meetings. The teams that gain are pentest and red team shops, for whom an agent that respects rules of engagement is the difference between a finding and an incident report. Most defenders will land in Daybreak Blue instead, where OpenAI is widening an alpha of GPT-6 Astra. The next signal is the Cyber at the Frontier livestream on November 12.
Introducing GPT‑6 Cyber: a major advance in security research that pairs stronger capabilities with better model alignment. The model powered the Linux kernel research we announced yesterday, and security teams at companies like Mozilla, Palo Alto Networks, Calif, and Cisco—alongside our own at OpenAI—are already putting it to work. GPT-6 Cyber is available to select Daybreak Red customers and we are expanding access over time. openai.com/index/introducing…
4
5
304
Anthropic let an outside evaluator into its internal models, and on Nov 5 that evaluator published that Anthropic’s own risk report had been overconfident. METR, with 2.5 full-time staff and high access since August, estimates Anthropic’s benchmark capability index (AECI) has grown 1.9x faster since October 2025 than in the March 2024 to October 2025 baseline, roughly 15 points a year rising to 29. Its guess is that about 80% of the current acceleration comes from AI doing AI R&D, and METR is confident faster compute scaling isn’t the driver. The internal model behind the jump, called Model 4 in the report, finished training in September and October and is the first to clear 50% on every benchmark it was tested on. The index is now grading on a curve it’s about to run off. The objection is that a partner auditing a lab is soft by design. Except in a separate review the same day, METR said the August risk report’s claim that the 2x threshold hadn’t been crossed was overstated, and that crossing it by July 15 is plausible. If you track Anthropic’s Responsible Scaling Policy, the signal to watch next is how the next system card reconciles its 1.7x estimate from the Haiku release with METR’s 1.9x.
1
63
Mistral’s pitch for Studio is sovereignty, and the part that actually delivers it, private cloud and on-prem deployment, doesn’t arrive until the end of November. On Nov 5 Mistral expanded Studio, its enterprise platform, with managed agents and apps in public preview. They run on Mistral Large 4, the 1.05T parameter open-weight model it calls Le Chonk. Agents get reusable skills, isolated code sandboxes, subagents and hooks, apps bundle hosting with auth and access controls, and workflows are now generally available. A new Mistral MCP server lets you build in Mistral Vibe or your existing coding agent and push to Studio with a single deploy command. For now, though, all of it runs on Mistral Cloud. The teams that gain most are European companies whose compliance people say no to sending data outside the perimeter, because agents, model and platform will be able to sit in their own VPC or server room. Tracing runs on OpenTelemetry, so if you already run an OTel stack, agent runs can land in the same dashboards as the rest of your services. Until the on-prem release ships, test it on Mistral Cloud with data your security team won’t lose sleep over. Le Chonk brings the brains, and in about three weeks on-prem brings the paperwork.
🆕 Le Chonk brings the brains. Mistral Studio gets it working. Introducing managed agents and apps, plus new capabilities for workflows, enterprise connections, observability and evaluations in Studio. Build faster. Deploy where you choose. Keep control. See what’s possible. 🧵
2
5
398
The line in Claude’s new Personal Finance launch that budgeting apps should read twice: no ads, no sponsored recommendations, and no credit cards or loans of its own. On Nov 5 Anthropic opened a US beta for Pro, Max and Ultra subscribers. It links bank and card accounts through Plaid, read-only, with up to two years of transactions. You ask in plain language where the money went, which bills are coming and which charges look odd, and Claude can’t move, manage or invest anything. ChatGPT got there first with a finance preview on May 15, but many personal finance apps earn on card and loan referrals, so Anthropic is betting the subscription can do that job instead. Somewhere an affiliate marketing team just scheduled an emergency meeting. The obvious objection is privacy, and the honest answer has two parts. Your bank password never reaches Claude and chats in the finance hub aren’t used for training. The transaction data still flows through Plaid and Anthropic to answer you, though, so read-only limits what can be done to your account but not who sees the data. If you’re in Europe, this stays a US-only demo for now.
Introducing Personal Finance in Claude, in beta in the US. Connect the accounts you choose, and Claude answers your money questions in everyday language, from your own balances and transactions. Access is read-only, and Claude can't move, manage, or invest your money.
1
83