Embeddings
Embeddings in the context of production machine-learning systems.
Lessons
Formal explanations, examples, simulations, and checkpoints.
PCA and representation geometry
PCA preserves dominant geometric variation, not automatically the information your decision needs.
Unsupervised learning, anomalies & retrievalClustering as an operational hypothesis
A cluster becomes knowledge only when stable geometry connects to a repeatable human or system action.
Unsupervised learning, anomalies & retrievalDensity clusters and novel anomalies
Density methods flag what is locally disconnected; people and outcomes determine what is truly novel and important.
Unsupervised learning, anomalies & retrievalEmbeddings and similarity
Similarity is a modeling choice: the same vectors tell different stories under cosine and dot product.
Unsupervised learning, anomalies & retrievalApproximate retrieval under latency
Approximate retrieval is an explicit recall–latency contract, not a transparent substitute for exact search.
Recommendation, search & rankingFrom corpus to slate: retrieve, score, re-rank
Diagnose recommendation quality stage by stage: coverage first, ordering second, slate policy third.
Recommendation, search & rankingClicks are not labels
A click is an exposed, policy-shaped outcome—not a universal relevance label.
Recommendation, search & rankingBM25, factorization, two towers, and ANN
Hybrid retrieval works because lexical, collaborative, and content representations fail in different places.
Recommendation, search & rankingPointwise, pairwise, and listwise ranking
Pointwise learns values, pairwise learns preferences, and listwise learns positions—choose the supervision that matches the slate.
Recommendation, search & rankingServing freshness, diversity, and experiments
A recommendation ships only when relevance, freshness, diversity, and system guardrails succeed together.
Build a transformer from scratchText to tensors: tokenization, embeddings, position
Tokenizer and position scheme are not preprocessing trivia; together they define what symbols and order the transformer can represent.
Build a transformer from scratchScaled dot-product self-attention by hand
Attention is a masked, normalized lookup: queries score keys, softmax allocates influence, and values carry the information.
Build a transformer from scratchHeads, norms, residuals, and MLPs
Attention mixes tokens, the MLP mixes channels, and normalized residual paths let both transformations accumulate without losing the original stream.
Courses & assignments
Dependency-authoritative learning units.
Unsupervised learning, anomalies & retrieval
A PCA, clustering, anomaly, and nearest-neighbor retrieval investigation.
Assignment: Incident discovery and retrieval systemCourse 08Recommendation, search & ranking
Candidate generation, ranking metrics, debiasing, and online experiment design.
Assignment: Hybrid catalog rankerCasebook
Reported facts and course reconstructions.