Authors:
(1) Sanchit Sinha, University of Virginia ([email protected]);
(2) Guangzhi Xiong, University of Virginia ([email protected]);
(3) Aidong Zhang, University of Virginia ([email protected]).
Table of Links
3 Methodology and 3.1 Representative Concept Extraction
3.2 Self-supervised Contrastive Concept Learning
3.3 Prototype-based Concept Grounding
3.4 End-to-end Composite Training
4 Experiments and 4.1 Datasets and Networks
4.3 Evaluation Metrics and 4.4 Generalization Results
4.5 Concept Fidelity and 4.6 Qualitative Visualization
4.5 Concept Fidelity
As RCE framework is explicitly regularized with a concept fidelity regularizer and grounded using prototypes, we would expect high fidelity scores. Table 5 lists the fidelity scores for the aforementioned baselines and our proposed method. Fidelity scores are averaged for each domain when taken as target (e.g. for domain (A) in DomainNet, the score is average of C→A, P→A and R→A). As expected, our method and BotCL, both with specific fidelity regularization outperform all other baseline approaches. Our method outperforms BotCL on most settings, except when the target domains are Art in DomainNet and Clipart in OfficHome due to siginificant domain dissonance.
4.6 Qualitative Visualization
Domain Alignment. We consider the extent to which the models trained using both concept grounding and contrastive learning maintain concept consistency not only within the source domain but also across the target domain as well. To understand what discriminative information is captured by a particular concept, Figure 5 shows the most important prototypes selected from the training set of both the source and target domains corresponding to five randomly selected concepts. We observe that prototypes explaining each concept are visually similar. For more results, refer Appendix.
Explanation using prototypes. For a given input sample, we also plot the prototypes associated to the highest activated concept, i.e., the important concept. Figure 6 shows the prototypes associated with the concepts most responsible for prediction (highest relevance scores). As can be seen, the prototypes possess distinct features, for eg., they capture round face of alarm clock. More results are reported in Appendix.
This paper is available on arxiv under CC BY 4.0 DEED license.
