The Effects of Low Prototype Counts on ML Model Interpretability and Similarity

Table of Links

Abstract and Intro

Background Information

Methodology

Prototype Interpretability

Prototype-query Similarity

Interpretability of the Decision-Making Process

The Effects of Low Prototype Counts

Discussions

The Effects of Low Prototype Counts

During preparation for our experiments, we noticed an unexpected phenomenon for prototype similarity. In some dataset/method combinations, it was possible to achieve high classification accuracies of more than 80% with only one prototype per class. For example, TesNet was able to achieve about 88% accuracy when trained on the ImageNet subset using only 7 prototypes. In this instance, the similarity of a prototype with its activation was very low. This sheds light on some of the limitations of these methods.

Part-prototype-based classifiers are, in the end, discriminative models. This is in contrast to how they are perceived from an interpretability standpoint. When humans look at the explanations given for the decision, they see the similarity between the prototypes and the activated regions. However such similarity is only useful to the model itself as long as it can utilize it to discriminate effectively between the classes. As long as a query sample is closer to the prototype of the correct class than all other classes, the model makes the correct decision, regardless of the actual similarity between the query sample and the prototype. When designing a model, setting the number of prototypes to a small value favors interpretability, but, on the other hand, it may adversely affect the similarity of the prototypes and their corresponding activations. Even when the number of prototypes is relatively small, to truly understand the model, it is essential to examine the similarity between prototypes and their corresponding activations (as in experiment 2), as well as the interpretability of the individual prototypes (as in experiment 1).