KV cache
What can a model safely reuse?
Caching follows from causality. Once you understand why an old state cannot see a newly appended token, you can derive the cache layout, position handling, and memory cost yourself.
The article
Related reading
02
04
Inside attention, one tensor at a time
Follow a complete decoder through Q, K, V, rotary positions, grouped-query attention, residual connections, and next-token logits. Every shape has a job.
Why modern attention has so many names
Separate head sharing, position, compressed memory, sparse selection, and GPU execution. Then read DeepSeek’s recent designs with those distinctions in mind.