DenseNet: Concatenating Every Previous Layer's Output

The problem, from first principles

A residual connection (04-modern-cnn-concepts/02-residual-connection) helps gradients flow by adding a layer's input to its output — but addition merges two feature maps into one, so whatever distinct information they each carried is now mixed together and can't be perfectly separated again downstream. DenseNet asks: what if, instead of merging, every layer's output were kept fully separate and simply concatenated onto a growing stack that every subsequent layer gets to see in full? Every layer then has direct, unmodified access to every earlier layer's exact feature maps, not just an additive blend of them.

From theory to code

Theory says: start with the input as the current "feature stack." For each layer, run a same-padded convolution and ReLU on the entire current stack (not just the most recent layer's output), then concatenate that layer's result onto the stack, growing it by that layer's output channel count before the next layer runs.

Implement dense_block(x, kernels) against that reasoning.

Constraints

• x: shape (C, H, W).
• kernels: a list of 3x3 kernels. kernels[i]'s input-channel count must equal the running total of channels accumulated so far (x's original channels plus every previous kernel's output channels).
• Every convolution uses padding 1 (spatial size never changes).
• Each layer's result is concatenated onto the front-preserving running stack along the channel axis — the original x always remains the first channels of the final output.
• An empty kernels list returns x unchanged.
• x and every kernel are never modified in place.

Hints

Open one at a time. Each gives away a little more than the last.