Scala Machine Learning Projects

Step 10 - Model evaluation on the highly-imbalanced data

Since the dataset is highly imbalanced towards non-fraudulent cases, using model evaluation metrics, such as accuracy or area under the curve (AUC), does not make sense. The reason is that these metrics would give overly optimistic results based on the high percentage of correct classifications of the majority class.

An alternative to AUC is to use the precision-recall curve, or the sensitivity (recall) -specificity curve. First, let's compute the ROC using the modelMetrics() method from the ModelMetricsSupport class:

val trainMetrics = ModelMetricsSupport.modelMetrics[ModelMetricsBinomial](model_nn_2, test)val auc = trainMetrics._aucval metrics = auc._tps.zip(auc._fps).zipWithIndex.map(x ...