Random Forest regression, and out-of-bag error estimation
Intermediate classical-mlensemblesrandom-forestregression
Implement:
def bootstrap_sample_with_oob(input, targets, seed=None) -> tuple[np.ndarray, np.ndarray, np.ndarray]:
"""Like 02-bagging's bootstrap_sample, plus the indices never drawn."""
def train_random_forest_regressor(input, targets, n_trees, max_depth, seed=None) -> list[tuple[dict, np.ndarray]]:
"""Returns [(tree, oob_indices), ...]."""
def predict_random_forest_regressor(forest, input) -> np.ndarray:
"""Averages every tree's prediction."""
def oob_error(forest, input, targets) -> float:
"""Mean squared error using only each sample's out-of-bag trees."""
- Reuse
05-regression-trees's build_regression_tree/predict_regression_tree.
predict_random_forest_regressor averages (regression), it does not vote (01-random-forest-majority-vote was for classification).