Minimal Module base class (parameter collection)

The problem, from first principles

Every optimizer question in this Part, sgd_step, adam_step, adamw_step, all take a flat list of params and a matching flat list of grads. A real network is never that flat: it's built by nesting layers inside layers inside layers, a Sequential container ([06-sequential-container], next in this track) might hold three Linear layers, each of which holds its own weight and bias. Somewhere, all of that nested structure needs to get flattened into the simple list the optimizer actually wants. Doing this by hand, walking every layer and manually collecting layer.weight, layer.bias, layer2.weight, ... for every layer in the network, would be repetitive and error-prone (easy to forget one layer), and would break the moment the network's architecture changed at all.

The standard solution, which every real deep learning framework uses in some form, is to give every layer a common base class that knows how to register its own parameters AND how to register child layers, then implement ONE recursive method, parameters(), that walks the whole nested structure and collects everything, no matter how deep the nesting goes.

From theory to code

Implement Module, a class with three pieces: __init__ sets up two empty dictionaries, self._parameters (name -> parameter array, for this module's OWN parameters) and self._modules (name -> child Module, for parameters that live one level deeper). register_parameter(name, value) and register_module(name, module) just store into those two dictionaries. parameters() does the actual work: return every value in self._parameters, PLUS every parameter returned by calling .parameters() on every child in self._modules (recursively, since a child could itself have children).

Constraints

• parameters() must return parameters from this module AND from every descendant, at any nesting depth, not just direct children.
• The order doesn't need to match any particular convention, but every registered parameter (from every level) must appear exactly once.
• register_parameter and register_module are separate: a Module should never be stored in self._parameters, and a raw array should never be stored in self._modules.

Hints