Concept Drift: The Relationship Between Inputs and the Target Shifting

The problem, from first principles

01-data-drift-psi catches a shift in WHAT inputs look like. But a model can keep seeing perfectly familiar-looking inputs while the actual RELATIONSHIP between those inputs and the correct answer changes underneath it: a spam filter's inputs might look statistically identical over time, but spammers constantly adapt their tactics, so what USED TO be a reliable spam signal stops being one. This is concept drift, and PSI can't catch it at all, since it never looks at labels or predictions: only actual PERFORMANCE monitoring can.

From theory to code

Implement sliding_window_accuracy (breaking a long prediction stream into windows to make a performance shift visible over time), detect_concept_drift, and first_drift_window.

Constraints

• sliding_window_accuracy(predictions, labels, window_size) splits the stream into consecutive, non-overlapping windows and returns each window's accuracy: a partial final window is dropped.
• detect_concept_drift(window_accuracies, baseline_accuracy, drop_threshold) returns True if ANY window's accuracy has fallen more than drop_threshold below baseline_accuracy.
• first_drift_window returns the INDEX of the first such window, or None if none qualify.

Hints

Open one at a time. Each gives away a little more than the last.