Sample Questions
Question 1
Easy
Random forests improve upon bagging by:
Solution
D is correct.
The key innovation of random forests over bagging is the random selection of a subset of predictors at each split point. This prevents dominant predictors from being used in every tree, thereby reducing the correlation between trees. Since the variance of an average of correlated quantities depends on the correlation, decorrelation leads to greater variance reduction.
The key innovation of random forests over bagging is the random selection of a subset of predictors at each split point. This prevents dominant predictors from being used in every tree, thereby reducing the correlation between trees. Since the variance of an average of correlated quantities depends on the correlation, decorrelation leads to greater variance reduction.
Question 2
Medium
In random forests, what role do fully grown (unpruned) trees play?
Solution
B is correct.
Random forests deliberately use fully grown, unpruned trees. Each individual tree has low bias (it fits the training data well, including noise) but high variance (it is very sensitive to the specific training data). The ensemble averaging across many trees reduces the variance while preserving the low bias. This is why random forests do not need pruning — the averaging handles the variance problem.
Random forests deliberately use fully grown, unpruned trees. Each individual tree has low bias (it fits the training data well, including noise) but high variance (it is very sensitive to the specific training data). The ensemble averaging across many trees reduces the variance while preserving the low bias. This is why random forests do not need pruning — the averaging handles the variance problem.
Question 3
Hard
Which statement about decision trees and outliers is correct?
Solution
E is correct.
Decision trees split by comparing predictor values to thresholds. The split only depends on whether an observation falls above or below the cutpoint, not on how far it is from the cutpoint. This makes trees inherently robust to outliers in the predictor space. An extreme value of a predictor simply falls to one side of a split just like any other value on that side.
Decision trees split by comparing predictor values to thresholds. The split only depends on whether an observation falls above or below the cutpoint, not on how far it is from the cutpoint. This makes trees inherently robust to outliers in the predictor space. An extreme value of a predictor simply falls to one side of a split just like any other value on that side.
