Residual (Skip) Connection

The problem, from first principles

Stacking more convolutional layers should let a network learn richer features — but past a certain depth, plain stacks of convolutions actually get harder to train, not easier: gradients have to flow back through every layer, and if a layer's Jacobian shrinks the gradient even slightly, dozens of layers compound that shrinkage until the earliest layers barely learn at all. A residual (skip) connection sidesteps this by adding a layer's input directly to its output, giving the gradient a direct, unobstructed path backward through the addition, regardless of how the convolution in between behaves.

From theory to code

Theory says: run the input through a convolution that is padded so its output has the exact same shape as the input, add the original input back to that convolution's output, then apply a nonlinearity. The network only has to learn a "residual" correction on top of the identity — if the best thing to do is nothing at all, it just has to learn to output all zeros from the convolution.

Implement residual_block(x, kernel) against that reasoning, reusing the existing multi-filter convolution and ReLU.

Constraints

• x: shape (C, H, W) — a single feature map, no batch dimension.
• kernel: shape (C, C, 3, 3) — same number of output channels as input channels, since the result must be addable back to x.
• Padding must be exactly 1 pixel on every side of H and W, so a 3x3 convolution leaves the spatial size unchanged.
• Returns an array the same shape as x.
• x and kernel are never modified in place.

Hints

Open one at a time. Each gives away a little more than the last.