If you learn AI Research Engineering, you will never be unemployed again. Stage 1: Tensor Math & Custom Autograd - Learn: PyTorch internals, custom forward/backward passes, Triton programming, memory layouts and stride calculations. - Practice: Write a custom fused attention kernel in Triton and a custom backward pass for a novel activation function without using PyTorch’s `autograd`. - Why: You cannot optimize or debug a collapsing loss curve if you treat neural networks as black-box APIs. Stage 2: Scaling Laws & Data Mixing - Learn: Chinchilla optimal compute, data deduplication (MinHash), domain-weighting and learning rate scheduling (Cosine with warm restarts). - Practice: Train a 100M parameter model on 3 different data-mix ratios (code vs. math vs. web text) and plot the exact loss divergence curves. - Why: In 2026, model architecture is secondary. The secret sauce is the exact mathematical ratio of the training data. Stage 3: Next-Gen Architectures (Beyond Transformers) - Learn: Mixture of Experts (MoE) routing, State-Space Models (Mamba/S4), linear attention and Ring Attention for infinite context. - Practice: Implement a sparse MoE routing layer with a load-balancing loss from scratch and benchmark its throughput against a dense transformer. - Why: The quadratic bottleneck of standard attention is being actively replaced. You must know the physics of the alternatives. Stage 4: Synthetic Data & Curation Pipelines - Learn: Self-instruct, evolutionary prompting, rejection sampling, and using frontier models to generate and filter training data. - Practice: Build a pipeline that uses a heavy reasoning model to generate 50,000 complex logic traces, filters them via a verifier and trains a smaller model on the survivors. - Why: The internet is running out of high-quality human text. The ability to mathematically curate synthetic data is the ultimate moat. Stage 5: Advanced Alignment & Preference Optimization - Learn: DPO (Direct Preference Optimization), KTO (Kahneman-Tversky Optimization), ORPO, reward modeling and mitigating reward hacking. - Practice: Align an SFT model using KTO (which only requires binary "good/bad" signals not paired preference data) and measure the shift in its behavioral distribution. - Why: SFT teaches a model what to say. Alignment teaches it how to behave without requiring millions of dollars in human annotation. Stage 6: Distributed Training & Systems ML - Learn: 3D Parallelism (Tensor, Pipeline, Data), DeepSpeed ZeRO-3, FSDP, NCCL communication primitives and gradient checkpointing. - Practice: Shard a 7B parameter model across 4 GPUs, profile the NCCL communication overhead and optimize the tensor-parallel boundaries. - Why: Single-GPU training is for tutorials. Production research requires sharding weights and optimizer states across clusters without bottlenecking the interconnect. Stage 7: Multimodal Fusion & Projection - Learn: Vision encoders (SigLIP, CLIP), cross-attention mechanisms, audio tokenization and early vs. late fusion architectures. - Practice: Train a lightweight MLP projection layer that maps SigLIP vision embeddings into an LLM's text embedding space to create a custom Vision-Language Model. - Why: Text is only half the internet. The 2026 meta is building models that natively reason over vision, audio and text simultaneously. Stage 8: Inference-Aware Model Design - Learn: Grouped Query Attention (GQA), Multi-Query Attention (MQA), KV-cache quantization and speculative decoding compatibility. - Practice: Modify a standard transformer to use GQA, measure the exact drop in KV-cache VRAM usage and benchmark the latency gain in vLLM. - Why: An MLE who trains a model that is too slow or expensive to serve has built a useless model. Architecture dictates inference physics. Stage 9: Mechanistic Interpretability - Learn: Activation patching, logit lens, circuit discovery and tools like TransformerLens or Baukit. - Practice: Locate the exact attention head responsible for a specific factual recall (e.g., "The Eiffel Tower is in...") and patch its activations to change the model's behavior. - Why: You cannot fix hallucinations or bias if you don't know which specific circuit in the neural network is generating them. Stage 10: Evaluation-Driven Development & Red Teaming - Learn: Contamination detection, dynamic evals, adversarial robustness and LLM-as-a-judge calibration. - Practice: Build an automated red-teaming agent that uses tree-of-thought prompting to systematically find jailbreaks in your safety classifier. - Why: Static benchmarks are easily gamed. You must build adversarial harnesses that continuously attack your own models. Stage 11: Paper-to-Production Prototyping - Learn: Rapid Arxiv reproduction, stripping academic fluff, identifying the "minimum viable kernel" of a new algorithm. - Practice: Take a brand-new Arxiv paper published this week, implement the core algorithm in <500 lines of PyTorch and benchmark it against the baseline. - Why: The speed at which you can translate academic math into working code is your primary value proposition to an AI lab. Stage 12: Open-Source Portfolio & Teardowns - Learn: HuggingFace ecosystem, writing rigorous model cards, publishing training logs and public benchmarking. - Practice: Publish a novel fine-tune, the exact synthetic data pipeline used to train it and a teardown of why the original paper's claimed results differed from your reproduction. - Why: Top AI labs hire Research Engineers based on their public HuggingFace repos and training methodologies not their degrees. Bookmark and Repost!

Congrats to @deepseek_ai on the launch of DeepInfer! 🎉 Great to see the engineering behind DeepSeek's production inference stack being open-sourced, and thanks for the shoutout to the SGLang community 🧡 We're especially excited about the work on large-batch and multi-GPU inference. Can't wait to dig deeper and bring some of these ideas into SGLang!

Open models. Open kernels. Open inference.💪

🌍 Open models. Open kernels. Open inference. From private deployments to public clouds, we’ll keep working with the community to make large-scale inference more efficient. Explore the code, contribute, and build with us! 4/4

What is AI really capable of in bio? To separate hype from reality, we spent 3 months working firsthand with @AnthropicAI's internal research model to see if it could create an experimental scorpion antivenom. Introducing Citrini Therapeutics. citriniresearch.com/p/citrin…

🚀 Introducing DeepInfer, our in-house inference framework, now fully open source! Powering DeepSeek-V4.1-Flash at scale with high throughput and low latency. We’re sharing our production stack to help the community lower compute costs, reduce energy use, and make AI more affordable. 🔗 github.com/deepseek-ai/DeepI… 1/4

great read for tonight anthropic.com/research/traci…

If you learn AI Infrastructure Engineering, you will never be unemployed again. Stage 1: Systems & GPU Foundations - Learn: C++, Rust, CUDA memory hierarchy, PCIe vs NVLink, thread blocks and warp scheduling. - Practice: Write a custom CUDA kernel for matrix multiplication from scratch and profile it against cuBLAS. Stage 2: Transformer Inference Physics - Learn: Prefill vs Decode phases, arithmetic intensity, memory bandwidth bottlenecks, KV-cache math. - Practice: Profile a HuggingFace model with Nsight Systems to identify the exact memory bottleneck during token generation. Stage 3: Modern Serving Engines & Batching - Learn: vLLM, SGLang, PagedAttention, continuous batching, chunked prefill. - Practice: Deploy a 70B model and tune chunk sizes to maximize throughput without starving decode requests during long-context prefills. Stage 4: KV-Cache & Memory Optimization - Learn: Prefix caching, KV quantization, CPU offloading, multi-turn cache reuse. - Practice: Build a routing proxy that directs requests with identical system prompts to the same replica to share KV blocks. Stage 5: Quantization & Compression - Learn: FP8, INT4, AWQ, GPTQ, Sparsity, TensorRT-LLM calibration. - Practice: Serve a model in FP8 vs FP16 and benchmark the exact perplexity drop against the latency and VRAM gains. Stage 6: Kernel-Level Engineering - Learn: Triton, FlashAttention, CUDA graphs, operator fusion. - Practice: Write a fused Triton kernel for RMSNorm or Softmax and benchmark it against the native PyTorch implementation. Stage 7: Distributed Inference & Parallelism - Learn: Tensor parallelism, Pipeline parallelism, Expert parallelism (for MoE models). - Practice: Shard a 405B parameter model across 8 nodes and measure the communication overhead between GPUs. Stage 8: Speculative Decoding - Learn: Draft-target models, Medusa heads, acceptance rates, n-gram drafting. - Practice: Build a pipeline where a local 3B model drafts tokens for a cloud 70B model to verify in parallel Stage 9: Multi-Node & Hardware Interconnects - Learn: NCCL, RDMA, InfiniBand, NVLink, Disaggregated Prefill/Decode architectures. - Practice: Set up a multi-node cluster and profile the network latency of tensor parallelism across nodes vs within a single node. Stage 10: Cluster Orchestration & GPU Scheduling - Learn: Kubernetes GPU operators, Ray, Slurm, MIG (Multi-Instance GPU) partitioning, KEDA. - Practice: Build a queue-based autoscaler that spins up spot GPUs based on pending inference requests and drains them when empty. Stage 11: AI Gateways, Routing & Observability - Learn: TTFT/ITL SLOs, semantic routing, DCGM metrics, OpenTelemetry for LLMs. - Practice: Build a gateway that routes simple queries to a quantized local model and complex reasoning to a frontier API based on prompt complexity. Stage 12: Public Benchmarks & Teardowns - Learn: Reproducible methodology, latency/throughput Pareto curves, cost-per-token analysis. Wrappers are a commodity. AI Infrastructure is the physics of scale. The modern AI Infra Engineer builds the silicon nervous system for global intelligence. Bookmark and Repost!

If you learn Forward Deployed Engineering, you will never be unemployed again. I’d do this. Stage 1: Enterprise Identity & Access (The Gatekeeper) - Learn: OAuth2, SAML, OIDC, SCIM provisioning, Row-Level Security (RLS) and multi-tenant data isolation. - Practice: Build an auth bridge connecting your AI app to a simulated customer's Okta/Azure AD, ensuring User A can only query data their specific RBAC role permits. - Why: No SSO and strict data isolation means no enterprise contract. Security review is the first gate. Stage 2: Air-Gapped & VPC Deployments (The Environment) - Learn: Kubernetes (K8s), Helm, Terraform, private container registries, air-gapped networking and egress filtering. - Practice: Script a one-click Terraform/Helm deployment of your entire AI stack into a simulated customer VPC with zero egress to the public internet. - Why: Banks, hospitals and governments don't let your AI talk to the public cloud. You must deploy in their basement. Stage 3: Legacy Data Extraction & ETL (The Fuel) - Learn: OCR, unstructured PDF parsing, mainframe DB connectors, Apache Airflow, dbt and data reconciliation. - Practice: Build a pipeline that ingests 10 years of messy, scanned customer invoices and turns them into a clean, queryable hybrid (vector + relational) store. - Why: Enterprise AI starves without clean data. The customer's data will always be a mess. Stage 4: Secure RAG & Permission-Filtered Retrieval (The Brain) - Learn: Hybrid search, document-level access control, citation grounding, metadata tagging and abstention logic. - Practice: Build a RAG pipeline where the LLM dynamically filters out retrieved chunks that the specific user asking the question doesn't have clearance to see. - Why: An AI that leaks HR salaries or M&A docs to a junior employee is an immediate breach of contract. Stage 5: MCP & Legacy System Integration (The Hands) - Learn: Model Context Protocol (MCP), SOAP/REST translation, web scraping, headless browsers and API rate-limit handling. - Practice: Build an MCP server that lets an LLM safely query and write to the customer's 15-year-old on-prem SAP or Oracle instance without exposing raw credentials. - Why: Agents are useless if they can't touch the customer's actual systems of record. Stage 6: AI Guardrails & PII Redaction (The Shield) - Learn: NER (Named Entity Recognition), deterministic regex filtering, LLM output validation, UUID masking, and audit logging. - Practice: Build a middleware proxy that intercepts LLM inputs/outputs, masks SSNs and credit cards with UUIDs before they hit the model, and restores them post-generation. - Why: Compliance teams will kill the deal if a single PII token hits an external frontier API. Stage 7: Human-in-the-Loop (HITL) Workflows (The Trust) - Learn: Temporal {dot} io, webhook routing, state machines, durable execution and UI approval gates. - Practice: Build a workflow where the AI drafts a $50k purchase order, but physically pauses execution until a human manager clicks "Approve" in Slack or Teams. - Why: Enterprises adopt AI only when humans remain legally accountable for the final action. Stage 8: Edge & On-Prem Inference (The Silicon) - Learn: vLLM, TensorRT-LLM, Ollama, GPU partitioning (MIG), and local quantized models (AWQ/GGUF). - Practice: Deploy a quantized 8B model on a single customer-owned GPU node to handle highly sensitive triage locally without network latency. - Why: Sometimes the data is too hot to move. You must bring the model to the data. Stage 9: Observability & Customer Health (The Pulse) - Learn: OpenTelemetry, Langfuse, custom Grafana dashboards, usage telemetry and cost-attribution. - Practice: Build a "Customer Health" dashboard showing the exact cost-savings, time-saved and token-spend the AI has generated for their specific team this month. - Why: If the customer doesn't see the ROI quantified, they will churn at renewal. Stage 10: Evals on Customer Data (The Proof) - Learn: Golden dataset creation, trajectory grading, shadow testing, and LLM-as-a-judge calibration. - Practice: Take 100 real past customer support tickets, run them through your agent in shadow mode, and prove to the VP of Support that it resolved 80% accurately. - Why: Generic benchmarks mean nothing to a buyer. You must prove the AI works on *their* messy data. Stage 11: Technical Discovery & Scoping (The Translation) - Learn: Stakeholder interviewing, architecture diagramming, SOW (Statement of Work) writing and expectation management. - Practice: Translate a vague "we want AI for supply chain" executive request into a scoped, 4-week technical deployment plan with defined, measurable success metrics. - Why: FDEs are the translators between revenue (Sales) and reality (Engineering). Stage 12: The Handoff & Self-Serve Admin (The Scale) - Learn: Technical writing, runbooks, admin portals, automated onboarding and chaos testing. - Practice: Write a deployment runbook and build an admin UI so the customer's internal IT team can manage the AI stack, update prompts and view logs without calling you. - Why: Your leverage is measured by how little the customer needs you for day-to-day operations. AI labs build the engine. FDEs build the roads. The modern Forward Deployed Engineer builds the secure, durable nervous system that connects frontier intelligence to legacy reality. Bookmark and Repost!

AI时代人成长的加速度的差距，大到可怕 一部分人，RSI和World Model反复絮叨，浮于表面，即使很年轻，但也显得有点登。 那些越深入这些事情的，踏实做过一些项目的同学，反倒更能平和的说出一些细节，本质，缺陷，等等 另一部分人，真是开始下工地，下工厂，拿数据，做infra，一个行业一个行业的啃落地，给AI或者物理AI扣细节，扣最后一公里。听之说话如沐春风 2波人，每隔一段时间，给人的感觉，真是差别太大了。还是更喜欢聊细节，做落地，激烈争辩，避免空对空宏大叙事

since agents have eaten the readme, i advocate for the humans.md

interpretability might be the most important scientific problem in the world right now, as I don't think we can solve alignment if we don't know what we're building. I think I have good news about it. I co-authored early work in mechanistic interpretability at OpenAI and came away pessimistic, because neural networks appeared to fold into arbitrary tangled knots. years later, I started openai's first large-scale automated interpretability project, finding about 2% of neurons legible with the primitive tools we had with modern agentic tools and a much closer look, the picture looks totally different. the vast majority in models up to trillions of parameters are legible, and their circuits beautiful. many famous examples of tangled neurons turned out to be misunderstood. The roadblocks I expected mostly aren't there. I now think ambitious interpretability will be solved by the time we need it most, and possibly quite soon, as automated scientists keep improving. it may be a bitter lesson for interpretability: the simple thing, at absurd scale, works already-clean.goodfire.com/

Inference engineering and contributing to VLLM and Sglang is cool, but it's also important to know how to scale your inference stack for a large amount of customers. This is where the market requires engineers, and I have written how to do so in a blog. Link in comments.

Give search over ALL of science to your agent ask.sciencemadereadable.com it's free it's open it's beautiful