Decision stump

Decision stump

A decision stump is a <a href="/en/Machine_learning" title="Machine learning" class="wl">machine learning</a> model consisting of a one-level <a href="/en/Decision_tree_learning" title="Decision tree learning" class="wl">decision tree</a>.<a href="#cite_note-IL92-1" style="counter-reset: mw-Ref 1;">[1]</a> That is, it is a decision tree with one internal node (the root) which is immediately connected to the terminal nodes (its leaves). A decision stump makes a prediction based on the value of just a single input feature. Sometimes they are also called 1-rules.<a href="#cite_note-2" style="counter-reset: mw-Ref 2;">[2]</a>

<div class="shortdescription nomobile noexcerpt noprint searchaux" style="display:none" about="#mwt1">Boolean classifier from one decision</div><figure class="mw-halign-right wiki-thumb"><a href="/en/File:Decision_stump.svg" class="mw-file-description image media-thumb" data-hash="File:Decision_stump.svg"><img loading="lazy" src="//upload.wikimedia.org/wikipedia/commons/thumb/c/cf/Decision_stump.svg/250px-Decision_stump.svg.png" decoding="async" data-file-width="241" data-file-height="87" data-file-type="drawing" height="90" width="250" srcset="//upload.wikimedia.org/wikipedia/commons/thumb/c/cf/Decision_stump.svg/375px-Decision_stump.svg.png 1.5x, //upload.wikimedia.org/wikipedia/commons/thumb/c/cf/Decision_stump.svg/500px-Decision_stump.svg.png 2x" alt="Decision_stump.svg"></a><figcaption>An example of a decision stump that discriminates between two of three classes of <a href="/en/Iris_flower_data_set" title="Iris flower data set" class="wl">Iris flower data set</a>: Iris versicolor and Iris virginica. The petal width is in centimetres. This particular stump achieves 94% accuracy on the Iris dataset for these two classes.</figcaption></figure>
Depending on the type of the input <a href="/en/Feature_(machine_learning)" title="Feature (machine learning)" class="wl">feature</a>, several variations are possible. For nominal features, one may build a stump which contains a leaf for each possible feature value<a href="#cite_note-3" style="counter-reset: mw-Ref 3;">[3]</a><a href="#cite_note-4" style="counter-reset: mw-Ref 4;">[4]</a> or a stump with the two leaves, one of which corresponds to some chosen category, and the other leaf to all the other categories.<a href="#cite_note-weka-5" style="counter-reset: mw-Ref 5;">[5]</a> For binary features these two schemes are identical. A missing value may be treated as a yet another category.<a href="#cite_note-weka-5" style="counter-reset: mw-Ref 5;">[5]</a>

For continuous features, usually, some threshold feature value is selected, and the stump contains two leaves — for values below and above the threshold. However, rarely, multiple thresholds may be chosen and the stump therefore contains three or more leaves.

Decision stumps are often<a href="#cite_note-6" style="counter-reset: mw-Ref 6;">[6]</a> used as components (called "weak learners" or "base learners") in <a href="/en/Machine_learning_ensemble" title="Machine learning ensemble" class="mw-redirect wl">machine learning ensemble</a> techniques such as <a href="/en/Bootstrap_aggregating" title="Bootstrap aggregating" class="wl">bagging</a> and <a href="/en/Boosting_(meta-algorithm)" title="Boosting (meta-algorithm)" class="mw-redirect wl">boosting</a>. For example, a <a href="/en/Viola–Jones_object_detection_framework" title="Viola–Jones object detection framework" class="wl">Viola–Jones</a> face detection algorithm employs <a href="/en/AdaBoost" title="AdaBoost" class="wl">AdaBoost</a> with decision stumps as weak learners.<a href="#cite_note-7" style="counter-reset: mw-Ref 7;">[7]</a>

The term "decision stump" was coined in a 1992 <a href="/en/International_Conference_on_Machine_Learning" title="International Conference on Machine Learning" class="wl">ICML</a> paper by Wayne Iba and Pat Langley.<a href="#cite_note-IL92-1" style="counter-reset: mw-Ref 1;">[1]</a><a href="#cite_note-8" style="counter-reset: mw-Ref 8;">[8]</a>

A decision stump is a machine learning model consisting of a one-level decision tree. Depending on the type of the input feature, several variations are possible. For nominal features, one may build a stump which contains a leaf for each possible feature value For continuous features, usually, some threshold feature value is selected, and the stump contains two leaves — for values below and above the threshold. However, rarely, multiple thresholds may be chosen and the stump therefore contains three or more leaves. 