Adaptive boosting based classification algorithm

AdaBoost

AdaBoost

AdaBoost, short for Adaptive <a href="/en/Boosting_(meta-algorithm)" title="Boosting (meta-algorithm)" class="mw-redirect wl">Boosting</a>, is a <a href="/en/Statistical_classification" title="Statistical classification" class="wl">statistical classification</a> <a href="/en/Meta-algorithm" title="Meta-algorithm" class="mw-redirect wl">meta-algorithm</a> formulated by <a href="/en/Yoav_Freund" title="Yoav Freund" class="wl">Yoav Freund</a> and <a href="/en/Robert_Schapire" title="Robert Schapire" class="wl">Robert Schapire</a> in 1995, who won the 2003 <a href="/en/Gödel_Prize" title="Gödel Prize" class="wl">Gödel Prize</a> for their work. It can be used in conjunction with many other types of learning algorithms to improve performance. The output of the other learning algorithms ('weak learners') is combined into a weighted sum that represents the final output of the boosted classifier. Usually, AdaBoost is presented for <a href="/en/Binary_classification" title="Binary classification" class="wl">binary classification</a>, although it can be generalized to multiple classes or bounded intervals on the real line.<a href="#cite_note-1" style="counter-reset: mw-Ref 1;">[1]</a><a href="#cite_note-2" style="counter-reset: mw-Ref 2;">[2]</a>

<div class="shortdescription nomobile noexcerpt noprint searchaux" style="display:none" about="#mwt1">Adaptive boosting based classification algorithm</div>
AdaBoost is adaptive in the sense that subsequent weak learners are tweaked in favor of those instances misclassified by previous classifiers. In some problems it can be less susceptible to the <a href="/en/Overfitting_(machine_learning)" title="Overfitting (machine learning)" class="mw-redirect wl">overfitting</a> problem than other learning algorithms. The individual learners can be weak, but as long as the performance of each one is slightly better than random guessing, the final model can be proven to converge to a strong learner.

Although AdaBoost is typically used to combine weak base learners (such as <a href="/en/Decision_stump" title="Decision stump" class="wl">decision stumps</a>), it has been shown that it can also effectively combine strong base learners (such as deep <a href="/en/Decision_tree" title="Decision tree" class="wl">decision trees</a>), producing an even more accurate model.<a href="#cite_note-3" style="counter-reset: mw-Ref 3;">[3]</a>

Every learning algorithm tends to suit some problem types better than others, and typically has many different parameters and configurations to adjust before it achieves optimal performance on a dataset. AdaBoost (with <a href="/en/Decision_tree_learning" title="Decision tree learning" class="wl">decision trees</a> as the weak learners) is often referred to as the best out-of-the-box classifier.<a href="#cite_note-4" style="counter-reset: mw-Ref 4;">[4]</a><a href="#cite_note-5" style="counter-reset: mw-Ref 5;">[5]</a> When used with decision tree learning, information gathered at each stage of the AdaBoost algorithm about the relative 'hardness' of each training sample is fed into the tree growing algorithm such that later trees tend to focus on harder-to-classify examples.

AdaBoost, short for Adaptive Boosting, is a statistical classification meta-algorithm formulated by Yoav Freund and Robert Schapire in 1995, who won the 2003 Gödel Prize for their work. It can be used in conjunction with many other types of learning algorithms to improve performance. The output of the other learning algorithms is combined into a weighted sum that represents the final output of the boosted classifier. Usually, AdaBoost is presented for binary classification, although it can be generalized to multiple classes or bounded intervals on the real line. AdaBoost is adaptive in the sense that subsequent weak learners are tweaked in favor of those instances misclassified by previous classifiers. In some problems it can be less susceptible to the overfitting problem than other learning algorithms. The individual learners can be weak, but as long as the performance of each one is slightly better than random guessing, the final model can be proven to converge to a strong learner. Although AdaBoost is typically used to combine weak base learners , it has been shown that it can also effectively combine strong base learners , producing an even more accurate model. 