Sometimes in data science and machine learning, we encounter problems of imbalanced classes. These are problems when one class might have more instances than another. This makes accuracy a bad metric. If class A shows up 90% in our sample and the other one 10%, then we can simply get 90% accuracy by always predicting class A. One metric that helps with this problem is (MCC), which was introduced in the binary setting by . Before we show the calculation for the MCC, let’s first revisit the concept of a confusion matrix. As you can see in the image below, a confusion matrix has four cells created by combining the predicted values against the real values. Two of those cells represent correct predictions (True Positives and True Negatives), and the other represents incorrect predictions (False Positives and False Negatives). Matthew’s Correlation Coefficient Matthews in 1975 Matthew’s correlation coefficient is calculated as follows: The MCC takes values between -1 and 1. A score of 1 indicates perfect agreement. But how does the MCC compare against other popular metrics for imbalanced classes? Matthew’s correlation coefficient vs. the F1-score The is another very popular metric for imbalanced class problems. The F1-score is calculated as: F1-score So, it is simply the of . According to a , the MCC has two advantages over the F1-score. harmonic mean precision and recall paper F1 varies for class swapping, while MCC is invariant if the positive class is renamed negative and vice versa. F1 is independent of the number of samples correctly classified as negative. Therefore, it is argued that the MCC is a complete measure compared to F1. Matthew’s correlation coefficient vs. Cohen’s Kappa is one of my favorite measures and one that we’ve written about on this blog. It’s a great metric for imbalanced class problems. This compares the two metrics. An argument is made in favor of the MCC, but I personally believe that it’s too theoretical. I’ve found that in practice both metrics give similar results, and I am using both in all my projects. Cohen’s kappa paper Using the MCC in Python and R Using the MCC in python is very easy. You can just use Scikit Learn’s metrics API. The MCC can be executed through the function . In R, you can use the function from the package. matthews_corrcoef mcc mltools Previously published . here

Handling Imbalanced Class Issues with Matthew’s Correlation Coefficient

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

3 Types of Anomalies in Anomaly Detection

Confusion Matrix in Machine Learning: Everything You Need to Know

Confusion Matrix Explained: The Real Foundation of Model Evaluation

What Are Convolution Neural Networks? [ELI5]

The Noonification: Have U Been Pwned? (1/12/2023)

Goldman Sachs, Data Lineage, and Harry Potter Spells

3 Types of Anomalies in Anomaly Detection

Confusion Matrix in Machine Learning: Everything You Need to Know

Confusion Matrix Explained: The Real Foundation of Model Evaluation

What Are Convolution Neural Networks? [ELI5]

The Noonification: Have U Been Pwned? (1/12/2023)

Goldman Sachs, Data Lineage, and Harry Potter Spells

Light-Mode

Classic

Newspaper

Minty

Dark-Mode

Neon Noir

Minty

HN StartUps