concept:prefill-decode
Prefill Decode
Prefill Decode in the context of production machine-learning systems.
Lessons
Formal explanations, examples, simulations, and checkpoints.
Frontier architectures & efficient inference
Autoregressive decoding policies
Use autoregressive decoding policies to move the efficient inference production brief toward a defensible release.
Frontier architectures & efficient inferenceKV caches, GQA/MQA, and long context
Use kv caches, gqa/mqa, and long context to move the efficient inference production brief toward a defensible release.
Courses & assignments
Dependency-authoritative learning units.
Casebook
Reported facts and course reconstructions.