Grafting (decision trees)

In machine learning, grafting is a technique for improving the classification accuracy of a decision tree. A decision tree is a model used to make predictions by following a flowchart-like structure of choices based on the data.

After an initial, simple tree is built from a set of training data, grafting carefully adds new decision points or "branches" to it. This process aims to increase the tree's predictive accuracy by refining its logic, especially in areas where the original tree made mistakes.

Grafting is the conceptual opposite of pruning, a more common technique where branches are removed from a complex tree to simplify it and prevent overfitting. While pruning makes a tree more general by reducing its complexity, grafting selectively adds complexity where it is predicted to improve results.

Illustration

The information required is given in the form of a chart as,

Information Chart

The nodes and leaves can be identified from the given information and the decision trees are constructed. One such decision tree is as follows,

Decision Tree branch for the information

Here the X-axis is represented as A and Y-axis as B. There are two cuts in the decision trees – nodes at 11 and 5 respective to A.

  A > 11
  A <= 11
  |  A >= 5
  |  A < 5

Using Grafting, new branches are added to the above classification.

Grafting Branches

Here B is also taken into consideration for the nodes and leaves. There are two more cuts at B – 7 and 2.

  A > 11
  A <= 11
  |  A >= 5
  |  A < 5
     |  B > 7
     |  B <= 7
        |  B > 2
        |  B <= 2

Thus the branching has increased due to the grafting technique.

This is the simplest form of illustration to represent grafting techniques.

Conclusion

Grafting can identify regions where there are no occupancy and correct the poor class assignments which increases the accuracy. The extension to graft multiple branches at each leaf reduces the number of errors.

However, the potential new branches have to be selected carefully to avoid increasing the error and failure cases.

Future Study

Improving multicast tree construction ^[1]

Problem of missing value in decision tree grafting ^[2] Optimal grafting and appropriate selection of branches to be added ^[3]

See also

• Decision tree
• Artificial neural network