XGBoost: The Optimal Leaf Weight
Beginner
The problem, from first principles
XGBoost (Chen & Guestrin, 2016) fits each tree by minimizing a second-order approximation of the loss. For a leaf, the best prediction depends only on the summed gradients and hessians of the samples that fall into it.
From theory to code
Implement xgb_leaf_weight(G, H, lam), which returns the optimal leaf value -G / (H + lam).
Constraints
lamis non-negative.
Hints
Hint 1
Negate the gradient sum and divide by the hessian sum plus the regularizer.
Keywords
research-papersclassical-mlboostingxgboost
Python
Run your code to see results
No results yet
Run your code to check it against the examples, or submit to run every test.