O'Reilly logo

Scala Machine Learning Projects by Md. Rezaul Karim

Stay ahead with the world's most comprehensive technology and business learning platform.

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, tutorials, and more.

Start Free Trial

No credit card required

Step 10 - Model evaluation on the highly-imbalanced data

Since the dataset is highly imbalanced towards non-fraudulent cases, using model evaluation metrics, such as accuracy or area under the curve (AUC), does not make sense. The reason is that these metrics would give overly optimistic results based on the high percentage of correct classifications of the majority class.

An alternative to AUC is to use the precision-recall curve, or the sensitivity (recall) -specificity curve. First, let's compute the ROC using the modelMetrics() method from the ModelMetricsSupport class:

val trainMetrics = ModelMetricsSupport.modelMetrics[ModelMetricsBinomial](model_nn_2, test)val auc = trainMetrics._aucval metrics = auc._tps.zip(auc._fps).zipWithIndex.map(x ...

With Safari, you learn the way you learn best. Get unlimited access to videos, live online training, learning paths, books, interactive tutorials, and more.

Start Free Trial

No credit card required