Hands-On Machine Learning for Algorithmic Trading

Defining a custom loss metric for early stopping

For binary classification, Area Under the Curve (AUC) is an excellent metric but is not provided by Keras. However, we can define a custom loss metric for use with the early stopping callback as follows (included in the preceding compile step):

def auc_roc(y_true, y_pred):    # any tensorflow metric    value, update_op = tf.metrics.auc(y_true, y_pred)    # find all variables created for this metric    metric_vars = [i for i in tf.local_variables() if 'auc_roc' in i.name.split('/')[1]]    # Add metric variables to GLOBAL_VARIABLES collection.    # They will be initialized for new session.    for v in metric_vars:        tf.add_to_collection(tf.GraphKeys.GLOBAL_VARIABLES, v)    # force to update metric values    with tf.control_dependencies([update_op]): ...