Canary Deployment: Rolling a New Model Out to a Small Slice of Traffic First
The problem, from first principles
03-ci-cd-for-ml's test suite can only catch problems that were anticipated ahead of time: real production traffic often surfaces issues no test suite predicted. Canary deployment is the safety net: route only a small, controlled percentage of REAL traffic to the new model, leaving the rest on the trusted stable version, so a problem shows up affecting a small fraction of users instead of everyone at once.
From theory to code
Implement hash_bucket (deterministic, uniformly-spread bucketing), is_routed_to_canary (the percentage-based routing decision), and route_request, tying the decision to an actual dispatch.
Constraints
hash_bucket(key, num_buckets)returns a deterministic bucket in[0, num_buckets), based on a real cryptographic hash ofkey.is_routed_to_canary(request_id, canary_percentage)uses 100 buckets: routes to the canary exactly when the request's bucket is less thancanary_percentage.- The SAME
request_idmust ALWAYS route the same way, at a givencanary_percentage: never a coin flip re-decided per call. route_requestdispatches to whichever model functionis_routed_to_canaryselects.
Hints
Open one at a time. Each gives away a little more than the last.
Hint 1
A cryptographic hash's output is effectively uniformly distributed across its whole range: reducing it modulo num_buckets spreads keys evenly across buckets, which is exactly what makes a hash-based router behave statistically like a real percentage split, without needing any actual randomness.
Hint 2
Using a REAL random number generator per-request (e.g. random.random() < percentage/100) would give the SAME request a DIFFERENT routing decision on a retry: a hash of the request's own ID is what makes routing decisions sticky and reproducible.
Keywords
No results yet
Run your code to check it against the examples, or submit to run every test.