Uncertainty
Uncertainty in the context of production machine-learning systems.
Lessons
Formal explanations, examples, simulations, and checkpoints.
Predict decisions, not just labels
A model score earns its place by improving a declared decision.
Problem framing, statistics & evaluationLabels, availability, and leakage
Train on what the system knew then, label with what became known later.
Problem framing, statistics & evaluationBaselines and honest splits
A baseline sets the floor; an honest split keeps the ruler straight.
Problem framing, statistics & evaluationMetrics, thresholds, and calibration
Measure ordering, repair probability meaning, then choose the policy the operation can actually run.
Problem framing, statistics & evaluationUncertainty and online experiments
Uncertainty tempers the estimate; randomization tests the cause; guardrails protect the product.
Problem framing, statistics & evaluationMetric selection by task and decision
Select metrics from the prediction target, decision, error costs, prevalence, slices, horizon, and deployment constraints—not from habit.
Forecasting & temporal MLForecast contracts and naive models
A forecast begins with two clocks—when you know and when you predict—and a baseline that respects both.
Forecasting & temporal MLTrend, seasonality, and known-future signals
A temporal feature is valid only if its value could be reconstructed exactly at the forecast origin.
Forecasting & temporal MLBacktesting without time travel
A faithful backtest is a replay of past information states, not merely a sorted split.
Forecasting & temporal MLStatistical and ML forecasters
Choose a forecaster and a multi-step strategy together; the estimator alone does not define the forecast.
Forecasting & temporal MLIntervals, drift, and forecast operations
Forecast reliability is the joint discipline of calibrated ranges, drift evidence, and a usable fallback.
Courses & assignments
Dependency-authoritative learning units.
Problem framing, statistics & evaluation
A metric contract, leakage-safe validation, calibration analysis, and experiment plan.
Assignment: Late-delivery evaluation contractCourse 07Forecasting & temporal ML
Rolling validation, statistical baselines, boosted forecasts, and uncertainty bands.
Assignment: Regional demand forecastCasebook
Reported facts and course reconstructions.