Flag risky listings, generate candidates, rank the slate, and choose review or fallback actions.
Cumulative capstone · 40–60 hours
Build a marketplace decision system that can survive production.
Combine seller and listing risk, listing text, product images, candidate retrieval, ranking, batch catalog scoring, and online serving. The model is only one artifact in the release.
3modalities
10milestones
2inference paths
1tested rollback
Production proposition
Bad catalog content harms trust; bad search hides good inventory.
Buyers need relevant, safe results. Sellers need fair exposure and actionable quality feedback. Operators need a reversible system inside explicit latency, review-capacity, and cost boundaries.
Cost-weighted risk, Recall@K, NDCG@K, coverage/diversity, calibration, p99, and cost per decision.
Nightly batch completes before inventory refresh; online search stays inside a declared p99 and queue limit.
False removals, unsafe listings, unequal exposure, stale inventory, privacy leakage, and expensive overload.
Decision contract & baselines
Define catalog-quality actions, search utility, temporal cutoffs, cost constraints, and deterministic tabular/text/image baselines.
- Decision and metric contract
- Heuristic and classical baselines
- Untouched temporal test
Data contracts & point-in-time datasets
Specify listing, seller, interaction, image, moderation, and outcome schemas with event/availability time, ownership, and replay.
- Schema and ownership registry
- Point-in-time joins
- Late-event and deletion policy
Multimodal representations
Build text and image representations, retrieve candidates, and document encoder/index compatibility.
- Embedding evaluation
- Exact-search recall baseline
- Encoder/index version contract
Risk and ranking models
Combine tabular quality risk, retrieval, learning-to-rank, availability, diversity, and cold-start behavior.
- Feature/model ablations
- NDCG and coverage slices
- Feedback-loop controls
Training & GPU plan
Implement repeatable training with a memory ledger, throughput profile, parallelism choice, and resumable checkpoints.
- Run manifests
- Memory/throughput profile
- Restore drill
Registry & release bundle
Package model, tokenizers, transforms, indexes, prompts/policies, datasets, evaluation, and environment as one promotable release.
- Artifact manifest
- Golden raw-input parity
- Provenance and model card
Batch and online inference
Design nightly catalog scoring plus online search with bounded queues, batching, caching, readiness, and fallback.
- Typed API/data contract
- Trace-driven load test
- Capacity and cost plan
Monitoring & experimentation
Join data, model, service, and decision telemetry; pre-register a controlled ranking launch and delayed-label review.
- SLIs/SLOs and alerts
- Experiment analysis plan
- Drift and retrain policy
Fairness, privacy & security review
Audit seller and buyer slices, retention, image/text risks, prompt injection, supply chain, human review, and appeal paths.
- Intersectional report
- Threat model
- Human oversight and recourse
Canary, rollback & postmortem
Run a simulated bad release, execute rollback, reconcile decisions, and write a blameless prevention-focused postmortem.
- Canary stop rules
- Rollback evidence
- Postmortem actions and owners
Final review packet
A reviewer can reproduce, challenge, operate, and reverse it.
Use synthetic or public data, but preserve production-shaped identities and cutoffs. Do not submit a hosted notebook as the system boundary.
Production self-review0/100
Staged hints
Failure diagnosis
- If offline numbers look impossible, audit prediction-time availability before model code.
- If quality is sound but the contract fails, measure the exact serialized and served path.
- If one slice regresses, preserve that slice as a permanent release gate.
Record an honest attempt to unlock the reference solution.