concept:fsdp
Fsdp
Fsdp in the context of production machine-learning systems.
Lessons
Formal explanations, examples, simulations, and checkpoints.
Distributed & fault-tolerant training
Distributed checkpoints and fault-tolerant restarts
Use distributed checkpoints and fault-tolerant restarts to move the distributed training production brief toward a defensible release.
Distributed & fault-tolerant trainingHangs, stragglers, and scaling efficiency
Use hangs, stragglers, and scaling efficiency to move the distributed training production brief toward a defensible release.
Courses & assignments
Dependency-authoritative learning units.
Casebook
Reported facts and course reconstructions.