02-sigmoid squashes its input into (0, 1) — always positive, which turns out to be a real practical downside for a hidden-layer activation: every unit downstream receives an always-positive signal, which tends to bias how gradients flow during training. Tanh is sigmoid's zero-centered cousin: the same S-shape, but squashed into (-1, 1) instead, so its output can be negative, positive, or exactly zero.
tanh(x) = 2*sigmoid(2x) - 1, so it's the same family of function as 02-sigmoid, and its derivative has the same "expressible purely in terms of the output" property. Implement tanh_forward(x) and tanh_backward(grad_output, output), taking output (the saved forward result), not x, same pattern as 02-sigmoid.
x: any NumPy array shape. tanh_forward returns the same shape, values in [-1, 1].tanh_backward(grad_output, output) takes the forward pass's own saved result, not the original x.tanh_forward must stay finite for any input, including very large |x|.Open one at a time. Each gives away a little more than the last.
NumPy already has this forward function built in — no need to derive it from exp by hand.
Tanh's derivative, like sigmoid's, is written purely in terms of its own output: f'(x) = 1 - f(x)^2.
Click "Run Tests" to test your implementation