THE TRANSFORMER TOKEN-THROUGHPUT AUDIT

Difficulty: Hard Tags: LLM Inference, Throughput, Batching, Systems

Story

An inference platform compares two serving configurations. Each request has a fixed per-request overhead plus a per-token cost; batching shares the overhead across requests but does not share token compute. Calculate average latency per request and aggregate throughput under the stated simplified model.

The Math

For batch size B, prompt-plus-generated token count per request T, fixed batch overhead a milliseconds, and per-token processing cost c milliseconds per token per request, batch latency is:

L = a + c*B*T

The batch completes B requests, so average latency per request is L/B, and aggregate throughput is 1000*B/L requests per second. This is a simplified cost model, not a claim about a particular serving engine.

Input Format

B (int), T (int), a (float), c (float).

Output Format

(batch_latency_ms, avg_latency_per_request_ms, throughput_rps) (6 decimal places).

Constraints

• 1 <= B, T <= 10^5, a, c > 0.
• Time limit: 1.0 second.

Example

Input

B = 4, T = 100, a = 10, c = 0.01

Output

(14.000000, 3.500000, 285.714286)

Explanation: The batch latency is 10 + 0.01*4*100 = 14 ms. Average latency is 14/4 = 3.5 ms, and throughput is 4000/14 ≈ 285.714286 requests/s.