MLE/ROADMAP
/Search
Menu
Learn/Course 08
Classical ML · weeks 10

Recommendation, search & ranking

Marketplace ranking learns from biased clicks and reinforces its own mistakes.

track:recommendation-searchfailure:feedback-loopconcept:embeddings
Opening production brief

A measurable problem, before a model.

Users

Operators, product owners, and people affected by the ranking systems decision.

Data

candidate generation, implicit feedback, exposure bias, two-tower embeddings with version, owner, event time, and availability contracts.

Constraints

Marketplace ranking learns from biased clicks and reinforces its own mistakes. Every solution must state latency, cost, capacity, and fallback limits.

Success

Offline evidence plus a deployable resolution: Candidate generation, ranking metrics, debiasing, and online experiment design.

CHAPTER 1

Frame and baseline

Turn the production problem into explicit data, metric, and baseline contracts.

  1. 01

    From corpus to slate: retrieve, score, re-rank

    Use from corpus to slate: retrieve, score, re-rank to move the ranking systems production brief toward a defensible release.

    65–90 min · checkpoint · deterministic lab
  2. 02

    Clicks are not labels

    Use clicks are not labels to move the ranking systems production brief toward a defensible release.

    65–90 min · checkpoint · deterministic lab
CHAPTER 2

Build and stress

Formalize the machinery, test counterexamples, and expose system limits.

  1. 03

    BM25, factorization, two towers, and ANN

    Use bm25, factorization, two towers, and ann to move the ranking systems production brief toward a defensible release.

    65–90 min · checkpoint · deterministic lab
  2. 04

    Pointwise, pairwise, and listwise ranking

    Use pointwise, pairwise, and listwise ranking to move the ranking systems production brief toward a defensible release.

    65–90 min · checkpoint · deterministic lab
CHAPTER 3

Resolve and operate

Join model behavior to architecture, observability, rollout, and rollback.

  1. 05

    Serving freshness, diversity, and experiments

    Use serving freshness, diversity, and experiments to move the ranking systems production brief toward a defensible release.

    65–90 min · checkpoint · deterministic lab
  2. HW

    Hybrid catalog ranker

    Specify BM25 plus learned retrieval, ranking, re-ranking, temporal evaluation, and a versioned online path.

    Rubric · staged hints · reference resolution
Prerequisite guidance

3 concepts make this faster.

Prerequisites are guidance, never hard gates.

C02EvaluationA metric contract, leakage-safe validation, calibration analysis, and experiment plan.C03Data & featuresData contracts, point-in-time features, backfills, quality gates, and lineage.C06Unsupervised & retrievalA PCA, clustering, anomaly, and nearest-neighbor retrieval investigation.
Ready to work the problem?

From corpus to slate: retrieve, score, re-rank

Start lesson 01