How does a large language model run on hardware? A large language model can feel like magic if your background is in CPU-centric programming. That's because the models combine deep neural network with the transformer architecture with a massively parallel processor, GPU, they run on. Furthermore, although you can express a large language model in a few hundred lines of Python, it calls into large, complex layers of software that are hard for a single person to grasp. To answer this question for myself, I implemented the Qwen3-0.6B model from scratch, from transformer to ISA simulator, with help from Claude Code as a weekend hack. That is, I implemented the model, the GPU instruction set, a compiler, and a GPU simulator to make every layer small enough for one person to understand deeply. I also have RTL for the GPU in the works to run the system on an FPGA, and perhaps in the future, tape it out to actual silicon just for fun. What's cool is that you can use it like Claude or Codex, but it runs on the CPU and is therefore fully debuggable and traceable. If you have had trouble connecting the dots between large language models and the GPUs they run on, check out the GitHub repository: github.com/penberg/titania

Went back and finished my Viewstamped Replication implementation in Rust. Protocol only, I/O is provided by the caller. Verified with a TigerBeetle-style simulator. Now thinking about doing some verification with Lean... github.com/penberg/vsr-rs