Note: CI/CD for ML, Testing a Model Like You Would Test Code Before It Ships

The problem, from first principles

02-training-pipelines-dag made training itself reproducible and automatable, but a pipeline that runs reliably can still produce a genuinely BAD model. Ordinary software CI blocks a deploy when tests fail; ML needs the same discipline, adapted to one real complication: a model's output can shift slightly between valid retrains (different random seed, slightly different data), so gating on "100% of tests pass" is often too strict to be useful in practice.

From theory to code

Implement run_model_tests (a real test-suite runner for a model), pass_rate, and gate_deployment, which decides whether a model is allowed to ship based on a PASS-RATE threshold rather than requiring every single test to succeed.

Constraints

• run_model_tests(model_fn, test_cases): each test case is (test_name, input_value, check_fn); a test that raises an exception counts as FAILED, never crashes the whole suite.
• pass_rate(results) returns the fraction of passing tests; an empty suite is vacuously 1.0.
• gate_deployment(results, min_pass_rate) returns True exactly when pass_rate(results) >= min_pass_rate.

Hints

Open one at a time. Each gives away a little more than the last.