THE PURPOSE OF THE DECISION TREES
 

 

 

 

 

 

 

 

 

 

 

 

 

Decision trees are powerful and popular for both classification and prediction. The attractiveness of tree-based methods is due largely to the fact that decision trees represent rules. Rules can readily be expressed in English so that we humans can understand them; they can also be expressed in a database access language such as SQL to retrieve records in a particular category. Decision trees are also useful for exploring data to gain insight into the relationships of a large number of candidate input variables to a target variable. Because deci­ sion trees combine both data exploration and modeling, they are a powerful first step in the modeling process even when building the final model using some other technique. There is often a trade-off between model accuracy and model transparency. In some applications, the accuracy of a classification or prediction is the only thing that matters; if a direct mail firm obtains a model that can accurately pre­ dict which members of a prospect pool are most likely to respond to a certain solicitation, the firm may not care how or why the model works.

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

 

In other situations, the ability to explain the reason for a decision is crucial. In insurance underwriting, for example, there are legal prohibitions against discrimination based on certain variables. An insurance company could find itself in the position of having to demonstrate to a court of law that it has not used illegal dis­ criminatory practices in granting or denying coverage. Similarly, it is more acceptable to both the loan officer and the credit applicant to hear that an application for credit has been denied on the basis of a computer-generated 165 166 Chapter 6 rule (such as income below some threshold and number of existing revolving accounts greater than some other threshold) than to hear that the decision has been made by a neural network that provides no explanation for its action. This with an examination of what decision trees are, how they work, and how they can be applied to classification and prediction prob­ lems. It then describes the core algorithm used to build decision trees and discusses some of the most popular variants of that core algorithm. Practical examples drawn from the authors’ experience are used to demonstrate the utility and general applicability of decision tree models and to illustrate prac­ tical considerations that must be taken into account.