concept:rope
Rope
Rope in the context of production machine-learning systems.
Lessons
Formal explanations, examples, simulations, and checkpoints.
Build a transformer from scratch
Scaled dot-product self-attention by hand
Attention is a masked, normalized lookup: queries score keys, softmax allocates influence, and values carry the information.
Build a transformer from scratchHeads, norms, residuals, and MLPs
Attention mixes tokens, the MLP mixes channels, and normalized residual paths let both transformations accumulate without losing the original stream.
Build a transformer from scratchTrain a causal language model
Causal LM training is next-token alignment plus a no-looking-ahead proof; everything else scales that contract.
Courses & assignments
Dependency-authoritative learning units.
Casebook
Reported facts and course reconstructions.