FlashAttention-2: Partitioning Work Into Blocks
Beginner
The problem, from first principles
FlashAttention-2 assigns thread blocks to query blocks so that GPU streaming multiprocessors stay busy. The number of such blocks determines how many can run in parallel.
From theory to code
Implement num_blocks(n, block), the ceiling of n divided by the block size.
Constraints
- Round up for a partial final block.
Hints
Hint 1
Use integer ceiling division.
Keywords
research-paperssystemsattentionkernels
Python
Run your code to see results
No results yet
Run your code to check it against the examples, or submit to run every test.