Python: Deeper Insights into Machine Learning

Evaluating and tuning the ensemble classifier

In this section, we are going to compute the ROC curves from the test set to check if the MajorityVoteClassifier generalizes well to unseen data. We should remember that the test set is not to be used for model selection; its only purpose is to report an unbiased estimate of the generalization performance of a classifier system. The code is as follows:

>>> from sklearn.metrics import roc_curve >>> from sklearn.metrics import auc >>> colors = ['black', 'orange', 'blue', 'green'] >>> linestyles = [':', '--', '-.', '-'] >>> for clf, label, clr, ls \ ... in zip(all_clf, clf_labels, colors, linestyles): ... # assuming the label of the positive class is 1 ... y_pred = clf.fit(X_train, ... y_train).predict_proba(X_test)[:, ...