Bayesian Optimization: Expected Improvement
Advanced
The problem, from first principles
Bayesian optimization (Frazier, 2018) tunes expensive functions, such as hyperparameters, by fitting a surrogate model with uncertainty and choosing the next point to try. Expected improvement scores each candidate by how much it is expected to beat the best result so far.
From theory to code
Implement expected_improvement(mu, sigma, best) for a minimization problem, using the standard normal CDF and density.
Constraints
sigmais positive.
Hints
Hint 1
Compute z = (best - mu) / sigma, then combine (best - mu) * Phi(z) with sigma * phi(z).
Keywords
research-papersclassical-mlbayesian-optimizationhyperparameters
Python
Run your code to see results
No results yet
Run your code to check it against the examples, or submit to run every test.