Precision, Recall, F1, ROC, AUC

The problem, from first principles

01-splitting-and-resampling gives a clean way to get held-out predictions; it doesn't say how to judge them. Plain accuracy hides a lot: on a dataset that's 95% negative, predicting "negative" for everything scores 95% accuracy while catching zero positives. A metric that only counts "right vs. wrong" can't tell that apart from a genuinely useful classifier.

What's actually needed is a way to look at the kinds of mistakes a classifier makes — how many real positives it misses, how many negatives it wrongly flags — and, since most classifiers output a score rather than a hard 0/1, a way to judge that whole range of possible decision thresholds at once rather than locking in just one.

From theory to code

Implement precision_recall_f1(labels, predictions) for a fixed set of hard predictions, then roc_curve(labels, scores) and auc(fpr, tpr) for the threshold-swept view built from continuous scores. Theory below derives the confusion-matrix counts and the curve; here the job is just counting matches with boolean masks and applying the resulting formulas.

Constraints

• labels, predictions in precision_recall_f1: {0, 1}-valued, same shape.
• Returns (precision, recall, f1), each 0.0 if its denominator would be 0 (no positive predictions for precision, no actual positives for recall) rather than raising or returning nan.
• scores in roc_curve: continuous, higher means "more likely class 1."
• roc_curve returns (fpr, tpr, thresholds), one entry per threshold tried: every distinct score value from highest to lowest, plus +inf (the threshold that classifies everything as 0).
• tpr/fpr at each threshold are computed against the total actual positive/negative counts, fixed for the whole dataset — not against the current threshold's predicted counts.
• auc(fpr, tpr) must not assume fpr already arrives sorted ascending.

Hints

Open one at a time. Each gives away a little more than the last.