Gradient Accumulation

Average a list of equally shaped micro-batch gradients element-wise. Gradient accumulation sums the gradients of several small batches before one optimiser step, imitating a large batch that would not fit in memory.

Implement solve(grads).

Returns. Return a NumPy array with the common shape of the gradients. An empty list or gradients of different shapes raise ValueError.

Examples

Example 1

Input:

solve([[1.0, 3.0], [3.0, 5.0]])

Output:

[2.0, 4.0]

Example 2

Input:

solve([[2.0, -2.0], [4.0, 0.0], [6.0, 2.0]])

Output:

[4.0, 0.0]