Machine Learning with R Cookbook - Second Edition

How it works...

The idea of boosting is to boost weak learners (for example, a single decision tree) into strong learners. Assuming that we have n points in our training dataset, we can assign a weight, Wi (0 <= i <n), for each point. Then, during the iterative learning process (we assume the number of iterations is m), we can reweigh each point in accordance with the classification result in each iteration. If the point is correctly classified, we should decrease the weight. Otherwise, we increase the weight of the point. When the iteration process is finished, we can then obtain the m fitted model, f_i(x) (0 <= i <n).

Finally, we can obtain the final prediction through the weighted average of each tree's prediction, where the weight, b, ...