concept:replay-tests
Replay Tests
Replay Tests in the context of production machine-learning systems.
Lessons
Formal explanations, examples, simulations, and checkpoints.
Reliability, monitoring & experimentation
Diagnose, mitigate, and write postmortems
Use diagnose, mitigate, and write postmortems to move the reliability production brief toward a defensible release.
Reliability, monitoring & experimentationRandomized experiments and guardrails
Use randomized experiments and guardrails to move the reliability production brief toward a defensible release.
Reliability, monitoring & experimentationDrift, retraining, rollback, continuous verification
Use drift, retraining, rollback, continuous verification to move the reliability production brief toward a defensible release.
Courses & assignments
Dependency-authoritative learning units.
Casebook
Reported facts and course reconstructions.