ML platform
Infrastructure and paved roads that let teams train, ship, observe, and govern models safely.
Lessons
Formal explanations, examples, simulations, and checkpoints.
From notebook to reproducible run
A reproducible result is a result whose complete input identity can be replayed.
Engineering foundationsContracts before models
Contracts keep bad meaning from becoming good-looking features.
Engineering foundationsTraining code is product code
The shipped model is the reloaded artifact, not the object still living in the trainer.
Engineering foundationsContainers and configuration without surprises
A container is reproducible only when its image, inputs, configuration, and outputs all have explicit identities.
Engineering foundationsObservable, idempotent, failure-safe workflows
A safe workflow makes commitment atomic and repetition uneventful.
Engineering foundationsRead a research paper as an executable specification
Turn a research paper into a testable claim map, implementation contract, and prioritized reading plan before writing code.
Engineering foundationsReproduce, validate, and productionize a paper
Implement a paper from baseline to novel component, diagnose reproduction gaps, and adapt it for production without silently changing its claim.
Data & feature foundationsData contracts, ownership, and lineage
A trustworthy dataset explains meaning, accountability, and ancestry—not just columns.
Data & feature foundationsPoint-in-time-correct features
Point-in-time correctness preserves the knowledge boundary of the original decision.
Data & feature foundationsFeature transformations that survive production
Ship the fitted representation contract with the model, in the same order and with the same edge cases.
Data & feature foundationsBatch, stream, and training-serving parity
Parity starts with one temporal meaning, then proves every execution path implements it.
Data & feature foundationsLabels, snapshots, and backfills
A label snapshot records what was knowable by a date, not the database’s latest opinion.
Distributed & fault-tolerant trainingChoose data, sharded, tensor, or pipeline parallelism
Parallelize the resource that actually exceeds its budget, and stop when the added communication costs more than the split saves.
Distributed & fault-tolerant trainingDDP under the hood
DDP is replicated local training held together by a strict collective contract: same parameters, matched backward work, partitioned data, averaged gradients.
Distributed & fault-tolerant trainingFSDP and ZeRO-style sharding
Sharding reduces steady replicated state, but wrap boundaries decide the temporary memory and communication you pay to reconstruct it.
Distributed & fault-tolerant trainingDistributed checkpoints and fault-tolerant restarts
Checkpoint correctness is next-step equivalence, protected by complete state, atomic publication, and rehearsed recovery.
Distributed & fault-tolerant trainingHangs, stragglers, and scaling efficiency
Distributed speed is synchronized speed: diagnose the first divergence, measure the slowest rank, and scale against quality and cost—not worker count.
Production data & ML platformsData contracts, ownership, lineage, and quality
A data contract is a shared operational promise about meaning, time, quality, and response—not a list of column types.
Production data & ML platformsLakehouse, batch, streaming, and event time
Reliable batch and streaming systems agree on event identity and semantics while staying honest about when information became knowable.
Production data & ML platformsFeature platforms and point-in-time correctness
Point-in-time correctness reconstructs what the system knew, not what the database knows now.
Production data & ML platformsOrchestration, metadata, experiments, and registries
Orchestration moves work; metadata and registry gates turn that work into reproducible, promotable evidence.
Production data & ML platformsPaved roads, multi-tenancy, and cost boundaries
A good platform standardizes safety and evidence while keeping workload-specific choices visible and reversible.
Courses & assignments
Dependency-authoritative learning units.
Engineering foundations
A seeded, tested, packaged, versioned training workflow with a model card.
Assignment: Reproducible baseline packageCourse 03Data & feature foundations
Data contracts, point-in-time features, backfills, quality gates, and lineage.
Assignment: Point-in-time feature systemCourse 13Distributed & fault-tolerant training
Efficient DDP, parallelism choices, elastic sampling, and resumable checkpoints.
Assignment: Fault-tolerant distributed trainerCourse 17Production data & ML platforms
A traced batch/stream platform with orchestration, registry, lineage, and ownership.
Assignment: Local mini ML platformCasebook
Reported facts and course reconstructions.
Michelangelo: one ML platform across the lifecycle
A useful ML platform is a connected lifecycle, not a pile of independent tools.
Airbnb · Data & platformChronon: feature definitions across offline and online paths
The feature definition is the product; stores and compute engines are implementations.
DoorDash · Data & platformA gigascale feature store with Redis
Online feature serving is a latency-critical database product with ML semantics.
Spotify · Data & platformThe winding road through TFX and Kubeflow
Platform adoption is a product and socio-technical problem.
Netflix · Data & platformMetaflow: a human-centric path from notebook to production
The best abstraction preserves the user’s reasoning model while adding production guarantees.
OpenAI · Training & distributionScaling Kubernetes to 7,500 nodes
At scale, metadata and coordination become a workload of their own.
Shopify · Serving & costA platform for real-time predictions
Online inference is a user-facing distributed system with a model inside.
Pinterest · Serving & costRay batch inference: coordinating data and model work
Batch inference is a bounded dataflow whose correctness includes exactly-once publication.
LinkedIn · Fairness, privacy & securityA fairness toolkit for large-scale AI
Fairness infrastructure must be context-aware, governed, and integrated into decisions.