Decision Tree in Machine Learning with example(Classification Problem)

Decision Tree in Machine Learning with proper example

Decision tree is a tree shaped diagram used to determine a course of action.Each branch of tree represents a possible decision, occurrence or reaction, it simply means possible solution to your decision based on certain conditions. 

Like, how do i determine a random vegetable from a shopping bag?

Advantages :

• Simple to understand, interpret and visualize
• Little effort is required for data preparation
• It can handle both categorical and numerical data
• Non-linear parameters don't effect its performance

Disadvantages:

• Overfitting occurs when the algorithm capture noise in the data
• The model can get unstable when they have small variations
• It is not ideal for larger dataset

Entropy:

It shows randomness in the dataset

S represents the dataset that entropy is calculated

c represents classes in the set

Entropy values fall between 0 and 1.

Information gain:

It measures decrease in entropy after the dataset is split, it simply means reduction in entropy

        Entropy(s) represents is the entropy of dataset

Gini index:

it simply measures impurity in your dataset

Let's Build our Decision Tree

Step 1: Compute the entropy for the dataset

Next, we have to select root node

Let's first consider outlook as our root node:

    On the outlook, there are three lables-sunny,overcast and rainy

In sunny, there are total 2 yes and 3 No's 

So, there are total 5

Likewise, we have to calculate for Overcast and Rainy

Now, we will select windy as a root node

Same as , we have to calculate for Temperature and Humidity

Now, we will select maximum gain as a root node

So, outlook is our ROOT node

Now, you have to calculate for the next branches....

We can also do this with GINI

Stay tuned to my next Blog

All the best👍

Thanks for reading💜

If you guys have any doubts or queries related about anything, Feel free to contact me.