most of those trillions of tokens on OpenRouter are not fresh work for GPUs, they’re agents repeatedly rereading the same convo OpenRouter’s daily numbers for the main Deepseek models sit around 10 trillion tokens. On the biggest one (V4.1 Flash), roughly 99% of that is input, and about 95% of the input is a cache hit. so that leaves something like: - 114 billion tokens the model actually generates - 460 billion tokens it has to process from scratch - 8.9 trillion tokens that are just the model being shown context it already saw earlier Agents do this constantly. Every turn they resend the entire session so far. The Openrouter counter counts all of it. -> but the GPU part is surprisingly small The fresh work (the 460B + 114B) is what actually needs compute. Spread over a full day that works out to a few million tokens per second. A single GB300 rack (so around 72 GPUs, roughly $7 million of hardware) can handle that load. So you could serve nearly all of Openrouter’s Deepseek traffic with one of these racks. where does the real cost shows up? Those 8.9 trillion cached tokens don’t rerun the model. They come back from Deepseek’s longer lived cache (something they started talking about in 2024). The model has already squeezed the stored state down to about 7 KB per token, yet the traffic still writes roughly 4 petabytes of new cache data every day. Keep just the last hour and you’re looking at around 165 TB (several times more than the high speed memory sitting inside the entire rack) In other words, the Openrouter meter is mostly measuring agents rereading their own history. It tells you less about GPU demand than it first appears, and more about the memory and storage sitting next to those GPUs. If you’re serving agents, the number that is the most important is cache footprint, not tokens per day. GPU compute for the fresh work is real but modest. What scales with agent traffic is the cache tier, the high speed memory plus the storage that feeds it. What would make this wrong: routing and caching get good enough that almost everything stays inside the GPUs’ own memory and the storage layer becomes a rounding error. But right now, that isn’t the case.