Speculative Decoding: Expected Accepted Tokens
Advanced
The problem, from first principles
Each speculative step yields the longest accepted prefix of the draft, plus one token from the target. If each draft token is accepted with probability alpha, the expected number of tokens per target pass is a geometric sum.
From theory to code
Implement expected_tokens_accepted(alpha, gamma), the expected tokens generated per target call.
Constraints
- Assume the acceptance events are independent with rate alpha.
Hints
Hint 1
Sum alpha to the power k for k from zero to gamma, which telescopes to the closed form.
Keywords
research-paperssystemsinferencedecoding
Python
Run your code to see results
No results yet
Run your code to check it against the examples, or submit to run every test.