What we can attribute directly
Shopify describes platform work supporting real-time machine-learning predictions.
The account covers model serving and the surrounding platform responsibilities.
Read Shopify Engineering — Real-time predictions Primary source · last checked 26 Aug 2026The operating envelope
Merchant-facing latency, high availability, model diversity, safe releases, and platform operability.
Model teams, platform owners, operators, downstream product systems, and people affected by decisions.
Versioned data, configs, traces, artifacts, deployments, and outcomes aligned on one timeline.
Teams can train models but lack a consistent way to expose predictions with production guarantees.
Trace the system before naming the bug.
- 01
Producers emit versioned data or model artifacts.
- 02
A platform validates, computes, stores, schedules, or routes them.
- 03
Training or inference consumes the exact declared version.
- 04
Telemetry joins the decision to system, data, and model identity.
- 05
Operators compare outcomes, stop conditions, and the last known-good path.
Follow the evidence boundary by boundary.
Symptoms
Teams can train models but lack a consistent way to expose predictions with production guarantees.
Investigation
Trace request validation, features, model lookup, runtime, logging, timeout, and fallback for a single decision.
A feature service exceeds its timeout. Specify the response, log record, and alert that preserve safety.
Open investigation scaffold
- Write the earliest known-bad timestamp.
- Compare exact identities on either side of that boundary.
- Find the smallest affected slice and a known-good counterexample.
- Separate mitigation from root-cause confirmation.
Repair the contract, not only the symptom.
Serving treated as model loading omits dependency, ownership, and degradation contracts.
Offer a typed serving path with versioned artifacts, readiness, observability, and controlled release.
Rollout
Start in shadow, canary by store or traffic slice, and require product fallback.
Reason about the tempting shortcuts.
- A single opaque endpoint for every latency class.
- Relying on infrastructure health without prediction telemetry.
Make recurrence visible early.
End-to-end and stage latency
Define owner, slice, normal range, alert persistence, and the exact mitigation the alert should trigger.
Model and feature version exposure
Define owner, slice, normal range, alert persistence, and the exact mitigation the alert should trigger.
Fallback and decision outcome
Define owner, slice, normal range, alert persistence, and the exact mitigation the alert should trigger.
Online inference is a user-facing distributed system with a model inside.
Carry this pattern into assignments as a design constraint and into incident reviews as a hypothesis—not as proof about an unpublished system.