Token Bucket Rate Limiting

The problem, from first principles

An autonomous agent can issue calls far faster than any person, so it needs a speed limit: on API calls, emails sent, files written. The token bucket is the standard way to enforce one that allows short bursts but caps the long-run rate. Picture a bucket holding up to capacity tokens. Each action removes one token (or more for expensive actions). Tokens drip back in at refill_rate per second. A full bucket allows a burst of capacity quick actions, and an empty bucket forces the agent to wait for tokens. As in the circuit breaker, time is passed in so that the behaviour is deterministic.

From theory to code

Implement the class TokenBucket.

Constraints

• TokenBucket(capacity, refill_rate) starts full at time 0.0.
• try_acquire(now, n=1): first refill by refill_rate * (now - last_time), capped at capacity, and set last_time = now. If at least n tokens are available, subtract n and return True; otherwise leave the tokens unchanged and return False.
• now never decreases between calls. tokens is an attribute holding the current (float) token count after the last call.

Hints