Normalized expected cost for a classifier

Summarize a classifier's cost over every operating point

Drummond and Holte (2006) showed that a classifier's error rates are only meaningful together with the class prior and the cost of each error. The normalized expected cost (NEC) turns a classifier at one operating point into a number between 0 and 1, where 0 is a perfect classifier and 1 is a trivial one.

With false negative rate FNR, false positive rate FPR, and the normalized positive-class cost fraction

PC = p * C_FN / (p * C_FN + (1 - p) * C_FP),

the NEC is NEC = FNR * PC + FPR * (1 - PC).

Implement normalized_expected_cost(fnr, fpr, pos_prior, cost_fn, cost_fp) and return the NEC as a float.

Constraints

• fnr and fpr are in [0, 1]. Otherwise raise ValueError.
• pos_prior is strictly between 0 and 1.
• cost_fn and cost_fp are strictly positive.
• Do not modify any inputs (they are scalars, so this only means no hidden state).

Hints