In the world of data science, metrics are the compass that guide our models to success. While many are familiar with the classic measures of precision and recall, there are actually a wide range of other options that are worth exploring.
In this article, we’ll dive into the Tversky index. This metric, a generalization of the Dice and Jaccard coefficients, can be extremely useful when trying to balance precision and recall against each other. When implemented as a loss function for neural networks, it can be a powerful way to deal with class imbalances.
A quick refresher on precision and recall
Imagine you are a detective tasked with capturing criminals in your town. In truth, there are 10 criminals roaming the streets.
In your first month, you bring in 8 suspects you assume to be criminals. Only 4 of them end up being guilty, while the other 4 are innocent.
If you were a machine learning model, you’d be evaluated against your precision and recall.
Precision asks: “of all those you caught, how many were criminals?”
Recall asks: “of all the criminals in the town, how many did you catch?”