Calculate P50, P95, and P99 Inference Latency

The problem, from first principles

Averages hide tail behavior: a service with a 50ms average latency can still have some users waiting 2 seconds. Given a list of per-request latency measurements, compute the P50 (median), P95, and P99 latency percentiles using linear interpolation between the two nearest ranks — the standard method reported by serving dashboards.

From theory to code

rank = p/100 * (n - 1)
percentile = sorted[floor(rank)] + (rank - floor(rank)) * (sorted[ceil(rank)] - sorted[floor(rank)])

Constraints

• Use the (n-1)-based linear interpolation rank formula, not the (n+1)-based or nearest-rank method.
• Handle n == 1 (all percentiles equal the single sample).
• Accept an arbitrary list of percentiles to compute (not hardcoded to just 50/95/99).

Hints