Cost per Success & Pareto Front

The problem, from first principles

The best agent on a leaderboard may be the one nobody can afford to run. A fair comparison puts cost next to quality. Cost per success divides the total spend by the number of successful tasks, so spending on failures counts against the agent. When comparing several configurations (different models, step budgets, prompts) it helps to find those that are not dominated: a configuration is dominated if another is at least as cheap and at least as successful, and strictly better in one of the two. The non-dominated set is the Pareto front, the only sensible choices, since every other configuration is beaten on both axes by something on the front.

From theory to code

Implement cost_per_success and pareto_front.

Constraints

• cost_per_success(costs, successes): costs is a list of per-task costs and successes a list of booleans of the same length. Return sum(costs) / (number of successes), or float('inf') if there are no successes.
• pareto_front(points): points is a list of (cost, success_rate) pairs. Point i is dominated if some other point j has cost_j <= cost_i and rate_j >= rate_i with at least one inequality strict. Return the sorted list of indices of non-dominated points. Identical duplicate points do not dominate each other.

Hints