Vanilla RNN cell, backward

The problem, from first principles

[01-rnn-cell-forward] computed h_next = tanh(x @ weight_ih.T + bias_ih + h_prev @ weight_hh.T + bias_hh), a composition of two linear transformations ([03-dl-training/02-layers/02-linear-backward] already derived the backward rule for exactly this kind of operation) followed by a tanh nonlinearity. This question chains those two known backward rules together: first backpropagate through tanh, then feed the result into TWO separate applications of the linear-layer backward rule, one for the input-to-hidden path, one for the hidden-to-hidden path.

The tanh backward rule has a genuinely convenient property worth calling out directly: d/dz tanh(z) = 1 - tanh(z)^2, expressed entirely in terms of tanh(z)'s OWN output, not the original input z. Since the forward pass already computed and returned h_next = tanh(z), the backward pass never needs to re-derive or re-access z at all, 1 - h_next**2 is directly computable from the SAVED forward output alone, a common and genuinely useful pattern (several other activation functions, like sigmoid, share this same "derivative expressible via the output" property).

From theory to code

Implement rnn_cell_backward(grad_h, h_next, x, h_prev, weight_ih, weight_hh). First, backpropagate through tanh: grad_z = grad_h * (1 - h_next**2). Then apply [03-dl-training/02-layers/02-linear-backward]'s linear_backward LOGIC twice with this same grad_z: once treating x/weight_ih as the linear layer's input/weight (producing grad_x, grad_weight_ih, grad_bias_ih), and once treating h_prev/weight_hh as the linear layer's input/weight (producing grad_h_prev, grad_weight_hh, grad_bias_hh).

Constraints

• grad_z must be computed via grad_h * (1 - h_next**2), using the ALREADY-COMPUTED forward output h_next, never recomputing tanh or its argument from scratch.
• grad_x = grad_z @ weight_ih and grad_h_prev = grad_z @ weight_hh (the SAME grad_z, applied through each path's own weight matrix).
• grad_weight_ih = grad_z.T @ x and grad_weight_hh = grad_z.T @ h_prev (again, the same grad_z, transposed and matrix-multiplied against each path's own input).
• grad_bias_ih and grad_bias_hh are BOTH grad_z.sum(axis=0), the SAME sum (since both biases were added to the SAME pre-activation z, each receives the identical gradient).

Hints