What we can attribute directly
Uber published D3 as an automated system to detect data drift.
The system addresses detection across datasets and feature distributions.
Read Uber Engineering — D3 Primary source · last checked 26 Aug 2026The operating envelope
Many models and features, seasonality, multiple comparisons, limited labels, alert fatigue, and ownership.
Model teams, platform owners, operators, downstream product systems, and people affected by decisions.
Versioned data, configs, traces, artifacts, deployments, and outcomes aligned on one timeline.
Silent input changes degrade models, while naive detectors generate too many unactionable alerts.
Trace the system before naming the bug.
- 01
Producers emit versioned data or model artifacts.
- 02
A platform validates, computes, stores, schedules, or routes them.
- 03
Training or inference consumes the exact declared version.
- 04
Telemetry joins the decision to system, data, and model identity.
- 05
Operators compare outcomes, stop conditions, and the last known-good path.
Follow the evidence boundary by boundary.
Symptoms
Silent input changes degrade models, while naive detectors generate too many unactionable alerts.
Investigation
Compare reference selection, sample volume, effect size, seasonality, slice behavior, and upstream changes.
Feature PSI rises after a holiday. What evidence separates expected seasonality from a broken pipeline?
Open investigation scaffold
- Write the earliest known-bad timestamp.
- Compare exact identities on either side of that boundary.
- Find the smallest affected slice and a known-good counterexample.
- Separate mitigation from root-cause confirmation.
Repair the contract, not only the symptom.
A statistically significant shift is not automatically a harmful or actionable production change.
Combine robust detectors with magnitude, persistence, model sensitivity, lineage, and routing to owners.
Rollout
Backtest on known changes, begin as a dashboard, then alert only on high-precision patterns.
Reason about the tempting shortcuts.
- Retraining on every drift signal.
- One universal threshold for every feature.
Make recurrence visible early.
Drift magnitude and persistence
Define owner, slice, normal range, alert persistence, and the exact mitigation the alert should trigger.
Alert precision and owner response
Define owner, slice, normal range, alert persistence, and the exact mitigation the alert should trigger.
Outcome quality when labels mature
Define owner, slice, normal range, alert persistence, and the exact mitigation the alert should trigger.
Drift detection is triage; impact and diagnosis decide the action.
Carry this pattern into assignments as a design constraint and into incident reviews as a hypothesis—not as proof about an unpublished system.