Regression trees: splitting on variance reduction instead of Gini
Intermediate classical-mldecision-treesregression
Implement, mirroring 01-gini-impurity through 03-best-split-minimal-tree for continuous targets instead of class labels:
def variance(targets: np.ndarray) -> float: ...
def variance_reduction(parent_targets, left_targets, right_targets) -> float: ...
def find_best_regression_split(input, targets) -> tuple[int, float, float] | None: ...
def build_regression_tree(input, targets, max_depth) -> dict: ...
def predict_regression_tree(tree, input) -> np.ndarray: ...
targets are real-valued, not class labels, np.array([2.3, -1.0, 4.7]) is a completely ordinary input.
- A leaf's prediction is the mean of its node's targets, not a majority vote, there's no "most common" real number.
predict_regression_tree returns a float array, predict_tree (from 03-best-split-minimal-tree) returns integer class labels, don't reuse it here, its output dtype would silently truncate real-valued predictions.