NLP / LLM
A focused route through sequence models, transformers, adaptation, evaluation, and language-model serving.
Lessons
Formal explanations, examples, simulations, and checkpoints.
Sequences are not IID
Use sequences are not iid to move the sequence models production brief toward a defensible release.
Sequence ML: RNNs, GRUs & LSTMsRecurrent state and BPTT
Use recurrent state and bptt to move the sequence models production brief toward a defensible release.
Sequence ML: RNNs, GRUs & LSTMsGates that remember: LSTM and GRU
Use gates that remember: lstm and gru to move the sequence models production brief toward a defensible release.
Sequence ML: RNNs, GRUs & LSTMsMany-to-one and many-to-many training
Use many-to-one and many-to-many training to move the sequence models production brief toward a defensible release.
Sequence ML: RNNs, GRUs & LSTMsStateful streaming inference without future leakage
Use stateful streaming inference without future leakage to move the sequence models production brief toward a defensible release.
Build a transformer from scratchText to tensors: tokenization, embeddings, position
Use text to tensors: tokenization, embeddings, position to move the transformer from scratch production brief toward a defensible release.
Build a transformer from scratchScaled dot-product self-attention by hand
Use scaled dot-product self-attention by hand to move the transformer from scratch production brief toward a defensible release.
Build a transformer from scratchHeads, norms, residuals, and MLPs
Use heads, norms, residuals, and mlps to move the transformer from scratch production brief toward a defensible release.
Build a transformer from scratchTrain a causal language model
Use train a causal language model to move the transformer from scratch production brief toward a defensible release.
Build a transformer from scratchSampling, KV cache, and efficient generation
Use sampling, kv cache, and efficient generation to move the transformer from scratch production brief toward a defensible release.
Courses & assignments
Dependency-authoritative learning units.
Sequence ML: RNNs, GRUs & LSTMs
A masked sequence model with BPTT, recurrent gates, and attention precursors.
Assignment: Causal stateful risk modelCourse 14Build a transformer from scratch
Tokenizer, attention, RoPE, residual stack, training loop, generation, and KV cache.
Assignment: Decoder-only transformer from scratchCasebook
Reported facts and course reconstructions.