GPipe: Micro-batch Size
Beginner
The problem, from first principles
Micro-batching splits the global batch so the pipeline can overlap work. Each micro-batch is smaller, and the number of them sets how full the pipeline gets.
From theory to code
Implement micro_batch_size(batch, micro), the size of each micro-batch.
Constraints
- Use integer division.
Hints
Hint 1
Floor-divide the batch by the number of micro-batches.
Keywords
research-paperssystemsparallelismpipeline
Python
Run your code to see results
No results yet
Run your code to check it against the examples, or submit to run every test.