concept:relu-gelu
Relu Gelu
Relu Gelu in the context of production machine-learning systems.
Lessons
Formal explanations, examples, simulations, and checkpoints.
Neural networks from scratch
Beyond linear boundaries
Hidden nonlinearities earn their place by making the final decision simpler in a learned representation.
Neural networks from scratchForward pass, activations, losses, and initialization
A trustworthy forward pass makes shapes, activation semantics, initialization, and loss expectations explicit.
Neural networks from scratchBackpropagation without magic
Backprop becomes ordinary once every edge in the forward graph has a local derivative you can inspect.
Courses & assignments
Dependency-authoritative learning units.
Casebook
Reported facts and course reconstructions.