## With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

No credit card required

# Chapter 6. Decision Trees

Like SVMs, Decision Trees are versatile Machine Learning algorithms that can perform both classification and regression tasks, and even multioutput tasks. They are very powerful algorithms, capable of fitting complex datasets. For example, in Chapter 2 you trained a `DecisionTreeRegressor` model on the California housing dataset, fitting it perfectly (actually overfitting it).

Decision Trees are also the fundamental components of Random Forests (see Chapter 7), which are among the most powerful Machine Learning algorithms available today.

In this chapter we will start by discussing how to train, visualize, and make predictions with Decision Trees. Then we will go through the CART training algorithm used by Scikit-Learn, and we will discuss how to regularize trees and use them for regression tasks. Finally, we will discuss some of the limitations of Decision Trees.

# Training and Visualizing a Decision Tree

To understand Decision Trees, let’s just build one and take a look at how it makes predictions. The following code trains a `DecisionTreeClassifier` on the iris dataset (see Chapter 4):

````from` `sklearn.datasets` `import` `load_iris`
`from` `sklearn.tree` `import` `DecisionTreeClassifier`

`iris` `=` `load_iris``()`
`X` `=` `iris``.``data``[:,` `2``:]` `# petal length and width`
`y` `=` `iris``.``target`

`tree_clf` `=` `DecisionTreeClassifier``(``max_depth``=``2``)`
`tree_clf``.``fit``(``X``,` `y``)````

You can visualize the trained Decision Tree by first using the `export_graphviz()` method to output a graph definition file called iris_tree.dot ...

## With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

No credit card required