new paper from USC, UC Irvine and Intel Labs on reusing a small model's prefill inside a big one: the small model reads the prompt, a linear map translates its KV cache into the big model's format, and the big model recomputes only part of the positions Qwen3-0.6B prefilling for Qwen3-14B (23x apart in size) with 30% of positions recomputed: GSM8K goes from 71.6% with the map alone to 90.2%, against 95.1% for the 14B on its own, and prefill runs up to 3.04x faster the map alone can leave the big model worse off than the small one: Llama 3.1 8B into Llama 3.3 70B scores 81.0%, below the 8B by itself (82.7%), and recomputing 40% of positions brings it to 92.0% section 2.2 is worth reading on its own: choosing which positions to recompute by attention score does worse than choosing at random, because the 14B puts 72% of its attention on the start token and 24% on the chat template and instruction, which leaves 4% for the actual question (at a 20% budget the attention-based selector doesn't pick a single question token) so attention weight mostly tells you where the sink and the template are arxiv.org/abs/2610.11358