PagedAttention: Blocks for a Sequence
Beginner
The problem, from first principles
vLLM (Kwon et al., 2023) stores the key-value cache in fixed-size blocks, like virtual memory pages, instead of one big contiguous buffer per request. A sequence claims as many blocks as its tokens need.
From theory to code
Implement vllm_num_blocks(seq_len, block_size), the number of blocks a sequence occupies.
Constraints
- Round up so a partly filled block is still allocated.
Hints
Hint 1
Use integer ceiling division by the block size.
Keywords
research-paperssystemsservingkv-cache
Python
Run your code to see results
No results yet
Run your code to check it against the examples, or submit to run every test.