Drift
Drift in the context of production machine-learning systems.
Lessons
Formal explanations, examples, simulations, and checkpoints.
Forecast contracts and naive models
Use forecast contracts and naive models to move the forecasting production brief toward a defensible release.
Forecasting & temporal MLTrend, seasonality, and known-future signals
Use trend, seasonality, and known-future signals to move the forecasting production brief toward a defensible release.
Forecasting & temporal MLBacktesting without time travel
Use backtesting without time travel to move the forecasting production brief toward a defensible release.
Forecasting & temporal MLStatistical and ML forecasters
Use statistical and ml forecasters to move the forecasting production brief toward a defensible release.
Forecasting & temporal MLIntervals, drift, and forecast operations
Use intervals, drift, and forecast operations to move the forecasting production brief toward a defensible release.
Reliability, monitoring & experimentationML SLIs, SLOs, error budgets, ownership
Use ml slis, slos, error budgets, ownership to move the reliability production brief toward a defensible release.
Reliability, monitoring & experimentationObserve system, data, prediction, and business
Use observe system, data, prediction, and business to move the reliability production brief toward a defensible release.
Reliability, monitoring & experimentationDiagnose, mitigate, and write postmortems
Use diagnose, mitigate, and write postmortems to move the reliability production brief toward a defensible release.
Reliability, monitoring & experimentationRandomized experiments and guardrails
Use randomized experiments and guardrails to move the reliability production brief toward a defensible release.
Reliability, monitoring & experimentationDrift, retraining, rollback, continuous verification
Use drift, retraining, rollback, continuous verification to move the reliability production brief toward a defensible release.
Courses & assignments
Dependency-authoritative learning units.
Forecasting & temporal ML
Rolling validation, statistical baselines, boosted forecasts, and uncertainty bands.
Assignment: Regional demand forecastCourse 19Reliability, monitoring & experimentation
Joined data/model/service telemetry, drift policy, experiments, alerts, and runbooks.
Assignment: Launch analysis and incident responseCasebook
Reported facts and course reconstructions.
D3: automated data-drift detection
Drift detection is triage; impact and diagnosis decide the action.
Cloudflare · Drift & incidentsMonitoring ML models for bot detection
Adversarial ML requires joined telemetry and active probes, not passive averages.
Stripe · Fairness, privacy & securityStripe Radar: learning fraud under adversarial pressure
Risk modeling connects delayed supervision, adversarial drift, calibrated decisions, and layered controls.