The Effects of Low Prototype Counts on ML Model Interpretability and Similarity

Authors: (1) Omid Davoodi, Carleton University, School of Computer Science; (2) Shayan Mohammadizadehsamakosh, Sharif University of Technology, Department of Computer Engineering; (3) Majid Komeili, Carleton University, School of Computer Science. Table of Links Abstract and Intro Background Information Methodology Prototype Interpretability Prototype-query Similarity Interpretability of the Decision-Making Process The Effects of Low Prototype Counts Discussions The Effects of Low Prototype Counts During preparation for our experiments, we noticed an unexpected phenomenon for prototype similarity. In some dataset/method combinations, it was possible to achieve high classification accuracies of more than 80% with only one prototype per class. For example, TesNet was able to achieve about 88% accuracy when trained on the ImageNet subset using only 7 prototypes. In this instance, the similarity of a prototype with its activation was very low. This sheds light on some of the limitations of these methods. Part-prototype-based classifiers are, in the end, discriminative models. This is in contrast to how they are perceived from an interpretability standpoint. When humans look at the explanations given for the decision, they see the similarity between the prototypes and the activated regions. However such similarity is only useful to the model itself as long as it can utilize it to discriminate effectively between the classes. As long as a query sample is closer to the prototype of the correct class than all other classes, the model makes the correct decision, regardless of the actual similarity between the query sample and the prototype. When designing a model, setting the number of prototypes to a small value favors interpretability, but, on the other hand, it may adversely affect the similarity of the prototypes and their corresponding activations. Even when the number of prototypes is relatively small, to truly understand the model, it is essential to examine the similarity between prototypes and their corresponding activations (as in experiment 2), as well as the interpretability of the individual prototypes (as in experiment 1). This paper is available on arxiv under CC 4.0 license. Authors: (1) Omid Davoodi, Carleton University, School of Computer Science; (2) Shayan Mohammadizadehsamakosh, Sharif University of Technology, Department of Computer Engineering; (3) Majid Komeili, Carleton University, School of Computer Science. Authors: Authors: (1) Omid Davoodi, Carleton University, School of Computer Science; (2) Shayan Mohammadizadehsamakosh, Sharif University of Technology, Department of Computer Engineering; (3) Majid Komeili, Carleton University, School of Computer Science. Table of Links Abstract and Intro Abstract and Intro Background Information Background Information Methodology Methodology Prototype Interpretability Prototype Interpretability Prototype-query Similarity Prototype-query Similarity Interpretability of the Decision-Making Process Interpretability of the Decision-Making Process The Effects of Low Prototype Counts The Effects of Low Prototype Counts Discussions Discussions The Effects of Low Prototype Counts During preparation for our experiments, we noticed an unexpected phenomenon for prototype similarity. In some dataset/method combinations, it was possible to achieve high classification accuracies of more than 80% with only one prototype per class. For example, TesNet was able to achieve about 88% accuracy when trained on the ImageNet subset using only 7 prototypes. In this instance, the similarity of a prototype with its activation was very low. This sheds light on some of the limitations of these methods. Part-prototype-based classifiers are, in the end, discriminative models. This is in contrast to how they are perceived from an interpretability standpoint. When humans look at the explanations given for the decision, they see the similarity between the prototypes and the activated regions. However such similarity is only useful to the model itself as long as it can utilize it to discriminate effectively between the classes. As long as a query sample is closer to the prototype of the correct class than all other classes, the model makes the correct decision, regardless of the actual similarity between the query sample and the prototype. When designing a model, setting the number of prototypes to a small value favors interpretability, but, on the other hand, it may adversely affect the similarity of the prototypes and their corresponding activations. Even when the number of prototypes is relatively small, to truly understand the model, it is essential to examine the similarity between prototypes and their corresponding activations (as in experiment 2), as well as the interpretability of the individual prototypes (as in experiment 1). This paper is available on arxiv under CC 4.0 license. This paper is available on arxiv under CC 4.0 license. available on arxiv