Backpropagation
Backpropagation in the context of production machine-learning systems.
Lessons
Formal explanations, examples, simulations, and checkpoints.
Beyond linear boundaries
Hidden nonlinearities earn their place by making the final decision simpler in a learned representation.
Neural networks from scratchForward pass, activations, losses, and initialization
A trustworthy forward pass makes shapes, activation semantics, initialization, and loss expectations explicit.
Neural networks from scratchBackpropagation without magic
Backprop becomes ordinary once every edge in the forward graph has a local derivative you can inspect.
Neural networks from scratchSGD, momentum, AdamW, and schedules
The trained state is parameters plus optimizer history, schedule position, data order, and randomness.
Neural networks from scratchRegularize, normalize, debug, and ship
Shipping a network means freezing its inference behavior, not merely saving its weights.
Neural networks from scratchActivation functions: geometry, gradients, and failure modes
Select activation functions by signal range, gradient flow, architecture, precision, and failure mode; then diagnose saturation and dead units from telemetry.
Neural networks from scratchLoss functions by task, data, and deployment constraints
Choose and debug training objectives as optimization surrogates for the real task, including imbalance, outliers, uncertainty, ranking, dense prediction, and generation.
Courses & assignments
Dependency-authoritative learning units.
Casebook
Reported facts and course reconstructions.