Operators, product owners, and people affected by the ml platforms decision.
Production · weeks 22
Production data & ML platforms
A late feature silently corrupts every downstream prediction.
Opening production brief
A measurable problem, before a model.
Kafka, Flink/Spark, Parquet/lakehouse, watermarks with version, owner, event time, and availability contracts.
A late feature silently corrupts every downstream prediction. Every solution must state latency, cost, capacity, and fallback limits.
Offline evidence plus a deployable resolution: A traced batch/stream platform with orchestration, registry, lineage, and ownership.
Frame and baseline
Turn the production problem into explicit data, metric, and baseline contracts.
- 01↗
Data contracts, ownership, lineage, and quality
A data contract is a shared operational promise about meaning, time, quality, and response—not a list of column types.
65–90 min · checkpoint · deterministic lab - 02↗
Lakehouse, batch, streaming, and event time
Reliable batch and streaming systems agree on event identity and semantics while staying honest about when information became knowable.
65–90 min · checkpoint · deterministic lab
Build and stress
Formalize the machinery, test counterexamples, and expose system limits.
- 03↗
Feature platforms and point-in-time correctness
Point-in-time correctness reconstructs what the system knew, not what the database knows now.
65–90 min · checkpoint · deterministic lab - 04↗
Orchestration, metadata, experiments, and registries
Orchestration moves work; metadata and registry gates turn that work into reproducible, promotable evidence.
65–90 min · checkpoint · deterministic lab
Resolve and operate
Join model behavior to architecture, observability, rollout, and rollback.
- 05↗
Paved roads, multi-tenancy, and cost boundaries
A good platform standardizes safety and evidence while keeping workload-specific choices visible and reversible.
65–90 min · checkpoint · deterministic lab - HW↗
Local mini ML platform
Specify idempotent events, a point-in-time join, shared feature definition, quality gates, lineage, and model registration.
Rubric · staged hints · reference resolution
Prerequisite guidance
2 concepts make this faster.
Prerequisites are guidance, never hard gates.