Random search, contrasted against grid search
The problem, from first principles
04-grid-search tries every combination in a fixed grid — exhaustive, but the grid itself has to be decided in advance, and its size multiplies across hyperparameters the same way a decision tree's split search grows combinatorially (03-best-split-minimal-tree). A learning rate grid of [0.01, 0.1, 1.0] will never test 0.347, no matter how good that value might be, because it was never in the list.
The fix doesn't need a smarter search order, just a different way of generating candidates: instead of a fixed, pre-listed grid, define a distribution to draw each hyperparameter from, and sample as many random configurations as the time budget allows, rather than however many a grid happens to contain.
From theory to code
Implement sample_params(param_distributions, rng) to draw one random configuration, then random_search(param_distributions, n_iter, fit_and_score_fn, seed=None) to repeat that n_iter times and track the best, mirroring 04-grid-search's grid_search bookkeeping.
Constraints
param_distributionsvalues are either a(low, high)tuple (continuous — sample uniformly in that range) or a list (discrete — sample one value uniformly from it).sample_paramsreturns one dict of hyperparameter values, one draw per key inparam_distributions.random_searchreturns the same shape as04-grid-search'sgrid_search:{"best_params", "best_score", "all_scores"}, withall_scoresholding exactlyn_iter(params, score)pairs.- Build exactly one
np.random.default_rng(seed)before the sampling loop — not one per iteration. - The same
seedmust reproduce identicalall_scoresandbest_paramsacross calls.
Hints
Open one at a time. Each gives away a little more than the last.
Hint 1
sample_params just needs to tell continuous from discrete specs apart: isinstance(spec, tuple) is the continuous case, anything else (a list) is discrete.
Hint 2
For continuous, rng.uniform(low, high) draws anywhere in the range. For discrete, don't reach for rng.choice — index the list with a random integer, spec[rng.integers(0, len(spec))].
Hint 3
random_search is structurally identical to grid_search, with one swap: instead of iterating a precomputed list of combinations, call sample_params fresh inside the loop, n_iter times, using the same rng object each time so the draws are independent, not repeated.
Keywords
No results yet
Run your code to check it against the examples, or submit to run every test.