free web hit counter

Introduction of Decision Trees

August 11, 2017
Print Friendly, PDF & Email

Decision Tree : Overview

Data mining techniques that help to make decisions using the available facts can be termed as Decision Tree. Decision trees are not only useful for decision making applications it is also used for classification and prediction task. There are some popular decision Tree algorithms namely C4.5, ID3, and CART. These algorithms are supervised learning algorithms. During training, the input sample patterns are represented as tree data structure.


An example of decision tree is given in figure 1, where nodes of tree shows the attributes of the data set. Additionally edges relationship among two nodes using the values of available attributes. The leaf node of the tree is recognized as decision node.

Decision-Trees example

Figure 1 decision tree example

Figure 1 demonstrates a decisions tree. Here decision labels are (yes or no), it is also known as class labels. In decision trees the class label is placed on leaf nodes. Additionally the nodes humidity, outlook and wind are attributes in data-set. Because the data set contains decision and attributes and decision tree graphically represent the data. Therefore it is helpful to understand the relationship among them. Sometimes the decision trees are used in form of IF THEN ELSE rules. An example of  IF THEN ELSE rule given as:

IF (Outlook = sun & Humidity = normal) then decision = yes


There are a number of utility and applications using the decision tree algorithms. The following key advantages of decision tree make it more acceptable:

  1. Decision tree are simple to understand and after a brief exploration we construct its.
  2. It requires modest data training. Earlier techniques demand for data normalization, creation of dummy variables and removal of blank values.
  3. It can deal with numerical as well as categorical data. Other techniques have their expertise into analysis of different sets of data having just single category of variables. Such as neural networks, that can handle only numerical values, while association rules only used with nominal variables.
  4. Utilizes the white box model. When a given condition is able to be seen in the model then rationalization for the provided scenario is easily elaborated by Boolean logic.
  5. Validation of representation with statistical tests is also possible. It makes the explanation for the reliability of the model possible.
  6. It is Robust and generates promising outcomes even when its postulations are debased to some extent with the actual model which generates the data.
  7. Time efficient even with large data. Standard computing resources help in analyzing large data.


  1. Decision-tree learning algorithms are based on heuristic algorithms which fail to offer an assurance of returning the globally optimal decision tree.
  2. It is possible that the decision tree learners can generate extra complex trees which may fail to generalize data properly. This is known as over fitting. For avoiding this, usage of some mechanisms such as pruning of data becomes necessary.
  3. Decision tree fails to describe some complex concepts like predicaments of XOR, parity or multiplexer. In such cases, large decision trees are generated. In order to overcome this issue two things can be done either shifting the depiction of the problem domain or by means of learning algorithms rooted on more communicative representations.
  4. For categorical variables that includes data with a number of levels, decision trees gives a preconception about the information gain on the side of the attributes with more levels.


[1]Susan Lomax, Sunil Vadera, “A Survey of Cost-Sensitive Decision Tree Induction Algorithms”, 2011, A survey of cost-sensitive decision tree induction algorithms ACM Computing Surveys, Article, 34 pages.

[2] Decision tree learning,

[3] Gregory Hamel, Advantages & Disadvantages of Decision Trees, advantages-disadvantages-of-decision-trees



Leave a Reply

Your email address will not be published. Required fields are marked *