ML Notes
Everything between a prompt and a token, one mechanism at a time: an interactive map of the LLM serving stack, from attention and the KV cache to agents and GPU kernels.
What it is
Notes on how large language models actually run, written the way I wish I had learned it. Each mechanism gets its own step-through page instead of a wall of text, and every page sits on one map of the stack, so you can see where it fits.
Four tracks
- Infrastructure — from prompt to token: prefill and decode, the KV cache, batching, and what happens when one GPU is not enough.
- Agents — the loop that wraps the model: context, tools and memory.
- Blackwell — one CUDA kernel, taken from 600 microseconds to 57.
- Foundations — from a single neuron to a trained network.