MLE/ROADMAP
/Search
Menu
Learn/Course 08
Classical ML · weeks 10

Recommendation, search & ranking

Marketplace ranking learns from biased clicks and reinforces its own mistakes.

track:recommendation-searchfailure:feedback-loopconcept:embeddings
Opening production brief

A measurable problem, before a model.

Users

Operators, product owners, and people affected by the ranking systems decision.

Data

candidate generation, implicit feedback, exposure bias, two-tower embeddings with version, owner, event time, and availability contracts.

Constraints

Marketplace ranking learns from biased clicks and reinforces its own mistakes. Every solution must state latency, cost, capacity, and fallback limits.

Success

Offline evidence plus a deployable resolution: Candidate generation, ranking metrics, debiasing, and online experiment design.

CHAPTER 1

Frame and baseline

Turn the production problem into explicit data, metric, and baseline contracts.

  1. 01

    From corpus to slate: retrieve, score, re-rank

    Diagnose recommendation quality stage by stage: coverage first, ordering second, slate policy third.

    65–90 min · checkpoint · deterministic lab
  2. 02

    Clicks are not labels

    A click is an exposed, policy-shaped outcome—not a universal relevance label.

    65–90 min · checkpoint · deterministic lab
CHAPTER 2

Build and stress

Formalize the machinery, test counterexamples, and expose system limits.

  1. 03

    BM25, factorization, two towers, and ANN

    Hybrid retrieval works because lexical, collaborative, and content representations fail in different places.

    65–90 min · checkpoint · deterministic lab
  2. 04

    Pointwise, pairwise, and listwise ranking

    Pointwise learns values, pairwise learns preferences, and listwise learns positions—choose the supervision that matches the slate.

    65–90 min · checkpoint · deterministic lab
CHAPTER 3

Resolve and operate

Join model behavior to architecture, observability, rollout, and rollback.

  1. 05

    Serving freshness, diversity, and experiments

    A recommendation ships only when relevance, freshness, diversity, and system guardrails succeed together.

    65–90 min · checkpoint · deterministic lab
  2. HW

    Hybrid catalog ranker

    Specify BM25 plus learned retrieval, ranking, re-ranking, temporal evaluation, and a versioned online path.

    Rubric · staged hints · reference resolution
Prerequisite guidance

3 concepts make this faster.

Prerequisites are guidance, never hard gates.

C02EvaluationA metric contract, leakage-safe validation, calibration analysis, and experiment plan.C03Data & featuresData contracts, point-in-time features, backfills, quality gates, and lineage.C06Unsupervised & retrievalA PCA, clustering, anomaly, and nearest-neighbor retrieval investigation.
Ready to work the problem?

From corpus to slate: retrieve, score, re-rank

Start lesson 01