Operators, product owners, and people affected by the responsible ml decision.
Production · weeks 24
Governance, fairness, privacy & security
A high-performing system creates subgroup harm and leaks sensitive information.
Opening production brief
A measurable problem, before a model.
intended use, equalized odds, intersectional slices, privacy threat modeling with version, owner, event time, and availability contracts.
A high-performing system creates subgroup harm and leaks sensitive information. Every solution must state latency, cost, capacity, and fallback limits.
Offline evidence plus a deployable resolution: A launch review covering fairness, privacy, licensing, abuse, security, and response.
Frame and baseline
Turn the production problem into explicit data, metric, and baseline contracts.
- 01↗
Inventory, risk tiers, accountability, evidence
Governance starts by giving the complete decision system a scope, an owner, a risk tier, and versioned evidence.
65–90 min · checkpoint · deterministic lab - 02↗
Fairness definitions, slices, thresholds, tradeoffs
Fairness work makes the chosen harm, comparison, uncertainty, and policy tradeoff explicit for every affected slice.
65–90 min · checkpoint · deterministic lab
Build and stress
Formalize the machinery, test counterexamples, and expose system limits.
- 03↗
Privacy minimization, retention, federated learning, DP
Privacy engineering reduces what the entire system can learn and retain about one protected unit over time.
65–90 min · checkpoint · deterministic lab - 04↗
Poisoning, evasion, extraction, and LLM threats
Models can propose; trusted code, identity, and policy must decide what the system is allowed to read, change, or reveal.
65–90 min · checkpoint · deterministic lab
Resolve and operate
Join model behavior to architecture, observability, rollout, and rollback.
- 05↗
Red teams, human oversight, audit, and response
Assurance is credible when adversarial testing, human intervention, auditability, and recovery are exercised against the exact system being released.
65–90 min · checkpoint · deterministic lab - HW↗
Model assurance package
Produce intended-use, fairness, privacy, security, human-review, monitored-rollout, and release evidence.
Rubric · staged hints · reference resolution
Prerequisite guidance
2 concepts make this faster.
Prerequisites are guidance, never hard gates.