Implementing McNemar's test
A more advanced statistical technique is McNemar's test. This test can be used on paired data to determine whether there are any differences between the two samples. As in the case of the t-test, we can use McNemar's test to determine whether two models give significantly different classification results.
McNemar's test operates on pairs of data points. This means that we need to know, for both classifiers, how they classified each data point. Based on the number of data points that the first classifier got right but the second got wrong and vice versa, we can determine whether the two classifiers are equivalent.
Let's assume the preceding Model A and Model B were applied to the same five data points. Whereas Model ...
Become an O’Reilly member and get unlimited access to this title plus top books and audiobooks from O’Reilly and nearly 200 top publishers, thousands of courses curated by job role, 150+ live events each month,
and much more.
Read now
Unlock full access