concept:cross-entropy
Cross Entropy
Cross Entropy in the context of production machine-learning systems.
Lessons
Formal explanations, examples, simulations, and checkpoints.
Neural networks from scratch
Forward pass, activations, losses, and initialization
A trustworthy forward pass makes shapes, activation semantics, initialization, and loss expectations explicit.
Neural networks from scratchBackpropagation without magic
Backprop becomes ordinary once every edge in the forward graph has a local derivative you can inspect.
Neural networks from scratchSGD, momentum, AdamW, and schedules
The trained state is parameters plus optimizer history, schedule position, data order, and randomness.
Courses & assignments
Dependency-authoritative learning units.
Casebook
Reported facts and course reconstructions.