Calibration
Calibration in the context of production machine-learning systems.
Lessons
Formal explanations, examples, simulations, and checkpoints.
Predict decisions, not just labels
A model score earns its place by improving a declared decision.
Problem framing, statistics & evaluationLabels, availability, and leakage
Train on what the system knew then, label with what became known later.
Problem framing, statistics & evaluationBaselines and honest splits
A baseline sets the floor; an honest split keeps the ruler straight.
Problem framing, statistics & evaluationMetrics, thresholds, and calibration
Measure ordering, repair probability meaning, then choose the policy the operation can actually run.
Problem framing, statistics & evaluationUncertainty and online experiments
Uncertainty tempers the estimate; randomization tests the cause; guardrails protect the product.
Problem framing, statistics & evaluationMetric selection by task and decision
Select metrics from the prediction target, decision, error costs, prevalence, slices, horizon, and deployment constraints—not from habit.
Trees, forests & gradient boostingTrees as learned decision rules
A tree learns readable local rules; constraints keep those rules from becoming a lookup table.
Trees, forests & gradient boostingRandom forests and diversity
A forest is strong when its trees are individually useful and collectively disagree in the right ways.
Trees, forests & gradient boostingGradient boosting as error correction
Boosting learns by repeatedly asking: what structured error remains?
Trees, forests & gradient boostingXGBoost: regularized boosting at scale
XGBoost scales boosting by turning gradient statistics into regularized decisions about leaves and splits.
Trees, forests & gradient boostingCalibrating and explaining ensembles
Calibrate probabilities for decisions, and read explanations as model accounting—not causal proof.
Courses & assignments
Dependency-authoritative learning units.
Problem framing, statistics & evaluation
A metric contract, leakage-safe validation, calibration analysis, and experiment plan.
Assignment: Late-delivery evaluation contractCourse 05Trees, forests & gradient boosting
Calibrated random-forest and XGBoost baselines with cost-sensitive thresholds.
Assignment: Rare-fraud ensemble decision systemCasebook
Reported facts and course reconstructions.