Mining Your Own Business in Telecoms Using DB2 Intelligent Miner for Data

94 Mining Your Own Business in Telecoms Using DB2 Intelligent Miner for Data

could be the criteria you use to generate the probable churners list and compare

to the list generated by the data mining model. If you don’t have a variable like

the contract expiration, then a randomly selected churners list could be used in

comparison.

5.6 Interpreting the results

In the previous section we looked at the steps to follow to get our mining results,

using different prediction mining techniques. The

sixth stage in our generic

mining method

is to interpret the results that we have obtained and determine

how we can map them to our business. When you are first confronted with

results, the first question to ask is “What does it all mean?”. In this section we

describe how to understand and interpret the results from the different

techniques.

5.6.1 Interpreting results from business perspective

The model was generated using decision tree, RBF and neural network

prediction

and compared using the gains chart. In the next section, we look into

the details from business perspective.

Decision Tree

The decision tree generates a tree model with a confusion matrix to see the

quality of the model, as shown in Figure 5-5.

Figure 5-5 Confusion matrix

Chapter 5. Can you predict the customers who are likely to leave? 95

In this case, the overall error was 26.12% which means that:

򐂰 1053 customers are churners: 523 are correctly classified and 530 are not.

򐂰 2967 customer are not churners: 2447 are correctly classified and 520 are

not.

򐂰 Overall 4020 customer: 1050 customers are incorrectly classified.

You can take an iterative process to improve the error rate of your tree model by

trying various subselections of your churn data model to build a tree and test it

using the test data set.

An acceptable error rate can be decided by verifying this model with other testing

data sets to see whether the error rate remains stable and also by your business

environment, for example, churn rate, and marketing campaign capacity.

In this case, actual churn rate is less than 5%, and as you can see the model

accuracy is 74% and accuracy among churners are up to 50% (1053 versus

523). Using the test data set, the error rate goes up slightly; however, the rate is

stable in several test data sets.

Graphical results of the tree model are displayed in Figure 5-6.

96 Mining Your Own Business in Telecoms Using DB2 Intelligent Miner for Data

Figure 5-6 Decision tree

Tree starts from the top and branches until it gets an optimum classification

result. There are leaf nodes at the bottom when it reaches to the optimum level

and have the customers who are classified by same rules.

Through the tree-like visualization of decision tree, you can see the classification

rules for the each leaf node and what most important variables are to build up a

rule. As you can see in Figure 5-6, OUTSPHERE is considered to be the most

important variable and then HANDSET, CUSTOMER RATE and so on, because

those variables are likely to appear in the top part of the tree.

Now, as an example, take a look at one leaf node which is classified as non

churners. As shown in Figure 5-7, according to the displayed rule, these are the

customers who are:

򐂰 Using three or more different call numbers for outbound calls (OUTSPHERE)

򐂰 Having an old model type of handset (HANDSET)

Chapter 5. Can you predict the customers who are likely to leave? 97

򐂰 Having either no contract obligation period from the beginning, or contract not

expired yet (CONTRACT_DUR).

򐂰 Having a high call success rate (CALL QUALITY)

This leaf node has 81.3% purity.

You can see on the HANDSET node in the tree, they have one node branching

left which has mostly non churners. These are customers using three or more

phone numbers for outbound; however, if their handset is new (HANDSET), then

they are more likely to stay in the company. This rule has 91.6% purity.

Figure 5-7 Decision tree with rule for non-churners

The decision tree for the churners is shown as an example in Figure 5-8.

According to the displayed rule, these are the customers whose first three rules

are the same as the non churners with the following differences:

򐂰 Low or medium call success rate