LeakyReLU fwd/bwd

The problem, from first principles

01-relu's backward pass returns exactly 0 for every x <= 0. That's fine for a single forward/backward pass, but consider what happens over many training steps: if a unit's input lands negative for every example it ever sees, its gradient is 0 on every one of those steps too, so nothing about its weights ever changes. The unit is permanently stuck, "dead" for the rest of training, no matter how much more data arrives. This is the well-known "dying ReLU" problem, and it's a direct consequence of ReLU's gradient having no way back once it hits zero.

LeakyReLU exists to give a negative-going unit an escape hatch: keep almost all of ReLU's behavior (positive inputs pass through unchanged, negative inputs shrink toward zero), but never let the gradient hit exactly zero on the negative side, so a unit that goes negative can still receive a small nudge and recover.

From theory to code

Theory's fix is a single number, negative_slope, that replaces ReLU's flat 0 region with a shallow line through the origin. Implement leaky_relu_forward(x, negative_slope) as this piecewise scaling, and leaky_relu_backward(grad_output, x, negative_slope) as its piecewise derivative, following the same "compute the local slope, then chain-rule multiply by grad_output" shape 01-relu's backward already uses.

Constraints

• x: any shape, elementwise operation.
• negative_slope: a scalar, defaults to 0.01, must be threaded through to both forward and backward.
• At x == 0, treat it as the x <= 0 branch: leaky_relu_forward returns 0, leaky_relu_backward returns negative_slope (the same convention 01-relu uses at its own kink, just with a different value on the negative side).
• One vectorized expression per function, no explicit loop over elements.
• Neither function modifies x or grad_output in place.

Hints

Open one at a time. Each gives away a little more than the last.