Speculative Decoding: The Acceptance Probability
Advanced
The problem, from first principles
Speculative decoding (Leviathan et al., 2023) lets a small draft model propose several tokens, which the large target model checks in one parallel pass. A drafted token is accepted with probability min(1, p over q), which keeps the output distribution exactly that of the target model.
From theory to code
Implement acceptance_prob(p, q), the acceptance probability of one drafted token.
Constraints
- Probabilities are in (0, 1].
Hints
Hint 1
Divide the target probability by the draft probability and cap the result at one.
Keywords
research-paperssystemsinferencedecoding
Python
Run your code to see results
No results yet
Run your code to check it against the examples, or submit to run every test.