Random forest classification

Random forest classification (Breiman 2001) in Past uses the same algorithm as decision trees except there is no pre- or post-pruning – each tree is grown to its tips. To decrease sensitivity to noise in the training set, a large number of randomized trees are generated, and classification is based on a majority vote among these trees. There are two levels of randomization:

1. The rows in the training set are sampled randomly with replacement (bootstrapped). This is also known as bagging.
2. During training, for every node, not all of the N variables are considered when searching for the optimal splitting rule. Instead, a number M < N of variables is selected at random. Setting M = N will disable this step. Typically, M is set to N/2 or smaller - √N is sometimes suggested.

Reference

Breiman, L. 2001. Random forests. Machine Learning 45:5–32.