Marquardt (@puercorum) profile picture
Joined September 2023
Photo by Marquardt (@puercorum)Photo by Marquardt (@puercorum)Photo by Marquardt (@puercorum)Photo by Marquardt (@puercorum)Photo by Marquardt (@puercorum)Photo by Marquardt (@puercorum)Photo by Marquardt (@puercorum)Photo by Marquardt (@puercorum)Photo by Marquardt (@puercorum)Photo by Marquardt (@puercorum)
Marquardt retweeted
Inference Engineering is becoming one of the most important skill sets in AI, and I think a lot of engineers are still underestimating it. Training gets most of the attention because that is where the model is created. But once a model has to serve real traffic, a different class of problems appears: queueing, batching, KV-cache pressure, GPU memory, scheduling, communication and tail latency. Inference is not one homogeneous workload. Prefill processes the prompt and builds the KV cache. Work across prompt positions can be parallelized efficiently, so prefill usually has higher arithmetic intensity and can use GPU compute. Decode is different. Each sequence advances autoregressively, usually one token per step, while repeatedly reading model weights and accumulated KV state. At practical serving batch sizes, decode can become heavily constrained by memory bandwidth. That split explains many inference optimizations. FlashAttention reduces memory traffic inside attention. PagedAttention manages KV-cache memory in blocks, reducing fragmentation. MQA and GQA reduce the number of key/value heads, cutting KV-cache storage and bandwidth. Continuous batching lets requests enter and leave the active batch as generation progresses. Chunked prefill breaks long prefills into smaller pieces so they can run alongside decode work. Prefix caching reuses KV state for repeated prefixes, saving prefill computation, but not making later decode tokens faster. Memory is another constraint. Model weights are only part of the footprint. For conventional attention, KV-cache size grows with active tokens and depends on layers, KV heads, head dimension and bytes per element. Quantization can reduce weight memory and bandwidth, while KV-cache quantization can reduce cache memory and traffic. Speculative decoding proposes tokens using a cheaper draft mechanism and verifies them with the target model. Speedup depends on draft cost, acceptance rate and verification efficiency. At larger scale, tensor parallelism adds communication, pipeline parallelism adds scheduling and bubbles, and MoE serving adds routing, communication and load-balancing problems. Some systems separate prefill and decode onto different GPU pools, paying the cost of KV transfer between them. This is why tokens per second is not enough. Production systems care about TTFT, ITL/TPOT, throughput, queueing delay, concurrency, KV-cache occupancy, GPU utilization, tail latency and goodput: how much request load still meets the latency objective. A server can show excellent aggregate throughput while users are still waiting too long for the first token. Inference engineering sits between ML systems, distributed systems, GPU architecture, networking, scheduling and performance engineering. The model decides what token should come next. Inference engineering determines how much serving that token costs, how quickly it reaches the user, and how many users the hardware can support at once. Happy Learning!!!
10
11
1
63
1,907
Marquardt retweeted
un buen flujo para construir software con agentes: 1) spec: definir qué querés construir y cómo debería funcionar. 2) research: explorar el codebase, entender la arquitectura y las dependencias antes de modificar código. 3) plan y orquestación: usar Opus para diseñar la solución, dividir tareas y coordinar agentes en paralelo. 4) implementación: delegar tareas a agentes con Sonnet. usar git worktrees para que trabajen de forma aislada. 5) verification loop: ejecutar tests, lint, typecheck y probar la UI con un browser. si algo falla, el agente corrige y vuelve a probar. 6) review: otro agente con Opus revisa el diff buscando bugs, regresiones y problemas de seguridad. si encuentra errores, vuelve a la etapa de implementación. con este flujo podés delegar tareas cada vez más complejas, aprovechar varios agentes trabajando en paralelo y reducir el tiempo que dedicás a supervisar cada cambio. me viene funcionando muy bien.
18
25
1
392
20,628
Marquardt retweeted
MOSFET Logic Gates – Part 2: NAND & OR ⚡ Continuing our deep dive into discrete MOSFET logic, this post breaks down the NAND and OR gates—covering how they operate, where current flows, and how to verify logic states with LEDs. 1️⃣ NAND Gate (NOT AND) 🔹 Inputs: 2+ → Output: 1 🔹 Logic: All inputs HIGH → Output LOW Any input LOW → Output HIGH 🔹 How it works: All HIGH → N‑MOS network conducts → Output LOW → LED off Input 1 LOW → Left P‑MOS conducts → Output HIGH → LED on Input 2 LOW → Right P‑MOS conducts → Output HIGH → LED on 💡 Key insight: Any LOW input turns on the corresponding P‑MOS, pulling the output HIGH. 🎯 Takeaway: Opposite of AND — PMOS pulls up, NMOS pulls down. 2️⃣ OR Gate 🔹 Inputs: 2+ → Output: 1 🔹 Logic: Any input HIGH → Output HIGH All inputs LOW → Output LOW 🔹 How it works: Any HIGH input → corresponding P‑MOS conducts → Output HIGH → LED on All LOW → N‑MOS conducts → Output LOW → LED off 💡 Key insight: Each P‑MOS acts independently to pull the output HIGH. 🎯 Takeaway: A “diligent” gate – reacts as soon as any input is HIGH. 📐 CMOS principle: PMOS for HIGH, NMOS for LOW. 📌 Design-for-Manufacturing (DFM) Tips Keep differential pair spacing consistent Minimize parasitic resistance on PMOS pull‑ups (critical for OR/NAND speed) Arrange layout for easy LED probing and visual debugging 🤔 Your turn: In mixed‑signal PCB designs, do you prefer testing logic gates with LEDs first (visual, quick sanity) or oscilloscope signals (precise, timing‑aware)? Let me know your go‑to method in the comments! 👇 #MOSFET #LogicGates #PCBLayout #CMOS #EmbeddedSystems #ElectronicsEngineering
22
1
146
2,872
Marquardt retweeted
Made a skill for stick figure explanations /𝚜𝚝𝚒𝚌𝚔-𝚜𝚌𝚎𝚗𝚎 [𝚙𝚛𝚘𝚖𝚙𝚝] Give it anything you want to visualize and it will generate the code and animation for you github.com/nilbuild/stick-sc…
47
144
9
1,807
72,603
Marquardt retweeted
I've been using AI coding agents for science (Claude, Gemini) a lot. Here are some tips on getting free access to frontier models, setting up autonomous agents, and links to discussions on its impact in math and theoretical sciences: sites.google.com/view/efgalv…
2
27
1
235
14,689
Marquardt retweeted
This is a good read from Anthropic on why you don't really need subagents anymore, and most workflows should just be one agent. Bitter lesson adjacent. Lots of good bits in here, both why this is true, and how you can convert back to a single agent architecture. Some quick notes on what I found interesting: - Modeling your agents around your org chart is *not* what you should do - Plan mode is dead. Better to have the plan be in the main agent's context - MCP tool design is critical. Take a jobs to be done approach versus mapping your API to tools directly. - Build evaluations on the full task completion / final output (or on the skills / tool calls if you need to). Whatever the case, make sure evals are coherent and non-conflicting - If you do keep subagents around, call it as a tool so that the main agent controls the context Also thematic in this post: Many attempts at AI adoption layer it on the existing architecture (both technical and organizational), when the most appropriate way to adopt it is to re-think the architecture completely. This is most difficult on the people side (e.g org design, what teams should do, what teams should even exist, etc.) Def. Worth reading the entire thing if you are doing applied AI stuff. claude.com/resources/article…
104
128
23
2,299
221,645
Marquardt retweeted
software factories feel like a temporary problem. the models will learn to build them for you the more interesting question to me is how to build a workspace where agents can collaborate with you
570
328
181
7,356
528,802
We built a tool for our own internal team at AnswerAI to let us learn, explore, & develop mastery with the help of AI: SolveIt. Today, we’re releasing it to the public for the 1st time. fast.ai/posts/2026-11-16-sol…
74
210
30
1,761
123,671
After 6 years in stealth, Aether's first paper is out. We designed and measured hundreds of proteins. The ones that leading models were most confident in were least likely to work: 84% didn't bind. A confident predicted shape is not evidence of function.
10
41
9
262
41,866
Marquardt retweeted
Claude writes better code when it can verify its work. Checks that used to be expensive, like tests, benchmarks, and end-to-end runs, are now cheap. Here’s how the Claude Code team thinks about verification: claude.dev/blog/how-the-clau…
363
873
247
14,660
1,697,751
Anthropic just told AI builders to delete most of their subagents. Their Applied AI team found that a single agent loop with skills and tools often beats complex multi-agent setups on quality, cost and speed. One company cut runtime roughly in half just by collapsing its multi-agent harness into one loop. Why multi-agent setups break down: • Teams copy their org chart into code (Conway's law in agent form) • Subagents play a lossy "telephone game" handing summaries back and forth • Every team patches the same system prompt into a conflicting Frankenprompt What works better now: • Skills: hold the playbook, rules and judgment calls (loaded only when needed) • Tools / MCP: handle live data, actions and permission checks • Subagents: save them only for parallel work, isolated context or real context bloat By loading skills and tools on demand instead of stuffing everything up front, Anthropic cut Claude Code's system prompt by over 80% on newer models. Their playbook to simplify yours: • Write a 10-example end-to-end eval first • Fold one subagent at a time into a skill + tools • Keep the subagent only if the eval drops Full Anthropic guide: claude.com/resources/article…
27
10
1
120
8,695
Understanding system design helps you build better apps, even when vibe coding. Better architecture decisions. Better AI prompts. Fewer surprises when your app grows. So I built an interactive app to learn system design visually. designed-to-scale.vercel.app
188
679
54
6,250
346,229
Marquardt retweeted
Mental models that runs almost 93% of engineering
45
1,444
27
8,945
319,963
Awesome Robotics! 💾 Building your own robot just got a lot easier. @MathWorks have released an open-source GitHub repository packed with robotics resources for anyone interested in getting hands-on. The repo includes examples for robot arms, ground vehicles, and drones, with projects that show how to connect with ROS and ROS2 or even deploy Simulink models directly as ROS nodes. There are also more advanced demos, like modeling off-road environments and testing navigation algorithms in photorealistic simulations. Everything is well-documented, with tutorials and links that make it easy to go from concept to prototype. 📑 Whether you’re a student, researcher, or just curious, there’s material here for every level. And since it’s an open community project, you can not only explore but also share your own contributions. For anyone looking to learn robotics by doing, this is a solid place to start! Here’s the link: github.com/mathworks-robotic… ~~ ♻️ Join the weekly robotics newsletter, and never miss any news → ziegler.substack.com
17
159
5
1,277
34,418
Marquardt retweeted
Anthropic tiene una guía de 36 páginas sobre cómo construir una startup AI-native en 2026. la divide en 4 etapas: 1) idea -> usar IA para investigar el mercado, analizar competidores y validar si realmente existe un problema que vale la pena resolver. 2) MVP -> construir rápido con claude code, documentar decisiones técnicas, mantener el contexto entre sesiones y medir cómo usan el producto los primeros usuarios. 3) launch -> preparar el producto para crecer. mejorar la arquitectura, seguridad, automatizar operaciones y empezar a medir métricas de negocio. 4) scale -> construir una ventaja competitiva con datos propios, conocimiento del negocio e integraciones que sean difíciles de replicar. hay una idea que me pareció interesante: cuando construir software se vuelve cada vez más fácil, decidir qué construir y entender a tus usuarios se vuelve mucho más importante. la guía está muy buena para quienes quieren pasar de escribir código a construir buenos productos. les dejo el PDF: assets.claude.com/503ee04b4a…
41
160
10
1,943
137,613
The biggest gap in GenAI engineering is no longer model knowledge. It's the ability to turn AI into reliable, scalable products. Here are 15 projects that can prove you have real engineering skills. Build real-time streaming, voice and vision systems, self-correcting RAG, multi-agent orchestration, and generative UI. Stop treating AI as a chatbot. Learn to build systems that can perceive, reason and execute. Then tackle production failures. Implement structured output enforcement, agent security, trace replay, trajectory testing and cost kill-switches. A smart agent that crashes, loops or leaks data is not a production-ready agent. Master performance next. Optimize KV-cache sharing, test models through shadow routing, build WebGPU fallbacks, and create feedback loops with DPO. Measure latency, reliability and cost. Intelligence means little if your system is too slow or expensive to use. Finally, publish real benchmarks and technical breakdowns. Pick a few projects, deploy them, expose their weaknesses and show how you fixed them. Most people learn to call AI models. Great engineers build the systems that make them useful. Most people watch tutorials. Builders ship systems.
5
1
17
903
Why do we divide by n-1 instead of n when we want to estimate the variance? This is Bessel's correction term, and it made no sense to me back in university when I first learned it. But I finally understood why! Got 2 minutes to spare? Here's the explanation!!
23
135
17
1,320
107,744
Nobody prepares you for what happens when you actually switch to multi-agent engineering workflows. You stop writing lines of code and start managing parallel threads of intelligence. Three things hit me almost immediately: 1.) This is clearly the future Once you watch three agents investigate a bug, write tests, and patch separate modules simultaneously while you review specs, going back to single-file typing feels like switching from an IDE back to Notepad. 2.) We are going to burn an absurd amount of inference One prompt used to be a quick autocomplete. Now a single feature spins up planners, workers, verifiers, and retries and suddenly your token bill looks like a small cloud cluster. 3.) It barely feels like coding, yet your brain is completely fried by 6 PM You aren't fighting syntax anymore. You're doing 100% pure architecture, constraint-setting, context management and edge-case review all day long. No boilerplate relief. Just non-stop high-level decisions. Software engineering isn't getting "lazy." The floor for typing just dropped to zero, while the ceiling for judgment and systems thinking went straight through the roof.
18
12
3
97
5,344