Embeddings
Embeddings in the context of production machine-learning systems.
Lessons
Formal explanations, examples, simulations, and checkpoints.
PCA and representation geometry
Use pca and representation geometry to move the unsupervised & retrieval production brief toward a defensible release.
Unsupervised learning, anomalies & retrievalClustering as an operational hypothesis
Use clustering as an operational hypothesis to move the unsupervised & retrieval production brief toward a defensible release.
Unsupervised learning, anomalies & retrievalDensity clusters and novel anomalies
Use density clusters and novel anomalies to move the unsupervised & retrieval production brief toward a defensible release.
Unsupervised learning, anomalies & retrievalEmbeddings and similarity
Use embeddings and similarity to move the unsupervised & retrieval production brief toward a defensible release.
Unsupervised learning, anomalies & retrievalApproximate retrieval under latency
Use approximate retrieval under latency to move the unsupervised & retrieval production brief toward a defensible release.
Recommendation, search & rankingFrom corpus to slate: retrieve, score, re-rank
Use from corpus to slate: retrieve, score, re-rank to move the ranking systems production brief toward a defensible release.
Recommendation, search & rankingClicks are not labels
Use clicks are not labels to move the ranking systems production brief toward a defensible release.
Recommendation, search & rankingBM25, factorization, two towers, and ANN
Use bm25, factorization, two towers, and ann to move the ranking systems production brief toward a defensible release.
Recommendation, search & rankingPointwise, pairwise, and listwise ranking
Use pointwise, pairwise, and listwise ranking to move the ranking systems production brief toward a defensible release.
Recommendation, search & rankingServing freshness, diversity, and experiments
Use serving freshness, diversity, and experiments to move the ranking systems production brief toward a defensible release.
Build a transformer from scratchText to tensors: tokenization, embeddings, position
Use text to tensors: tokenization, embeddings, position to move the transformer from scratch production brief toward a defensible release.
Build a transformer from scratchScaled dot-product self-attention by hand
Use scaled dot-product self-attention by hand to move the transformer from scratch production brief toward a defensible release.
Build a transformer from scratchHeads, norms, residuals, and MLPs
Use heads, norms, residuals, and mlps to move the transformer from scratch production brief toward a defensible release.
Courses & assignments
Dependency-authoritative learning units.
Unsupervised learning, anomalies & retrieval
A PCA, clustering, anomaly, and nearest-neighbor retrieval investigation.
Assignment: Incident discovery and retrieval systemCourse 08Recommendation, search & ranking
Candidate generation, ranking metrics, debiasing, and online experiment design.
Assignment: Hybrid catalog rankerCasebook
Reported facts and course reconstructions.