Large Minibatch SGD: Workers per Batch
Beginner
The problem, from first principles
Large-batch training spreads a minibatch over many workers, each handling a slice. The number of workers needed depends on the global batch and how many examples fit on each worker.
From theory to code
Implement workers_for_batch(batch, per_worker), the number of workers needed to cover the batch.
Constraints
- Round up so the whole batch is covered.
Hints
Hint 1
Divide the batch by the per-worker count and take the ceiling.
Keywords
research-papersoptimizationlarge-batchdistributed
Python
Run your code to see results
No results yet
Run your code to check it against the examples, or submit to run every test.