lifecycle:monitoring

Monitoring

Monitoring in the context of production machine-learning systems.

Lessons

Formal explanations, examples, simulations, and checkpoints.

Use ml slis, slos, error budgets, ownership to move the reliability production brief toward a defensible release.

Use observe system, data, prediction, and business to move the reliability production brief toward a defensible release.

Use diagnose, mitigate, and write postmortems to move the reliability production brief toward a defensible release.

Use randomized experiments and guardrails to move the reliability production brief toward a defensible release.

Use drift, retraining, rollback, continuous verification to move the reliability production brief toward a defensible release.

Dependency-authoritative learning units.

Joined data/model/service telemetry, drift policy, experiments, alerts, and runbooks.

Assignment: Launch analysis and incident response

Reported facts and course reconstructions.