Table of Links Abstract and 1 Introduction Abstract and 1 Introduction Related works


Problem setting


Methodology
4.1. Decision boundary-aware distillation
4.2. Knowledge consolidation


Experimental results and 5.1. Experiment Setup
5.2. Comparison with SOTA methods
5.3. Ablation study


Conclusion and future work and References Related works Related works Related works Problem setting Problem setting Problem setting Methodology
4.1. Decision boundary-aware distillation
4.2. Knowledge consolidation Methodology Methodology 4.1. Decision boundary-aware distillation 4.1. Decision boundary-aware distillation 4.2. Knowledge consolidation 4.2. Knowledge consolidation Experimental results and 5.1. Experiment Setup
5.2. Comparison with SOTA methods
5.3. Ablation study Experimental results and 5.1. Experiment Setup Experimental results and 5.1. Experiment Setup 5.2. Comparison with SOTA methods 5.2. Comparison with SOTA methods 5.3. Ablation study 5.3. Ablation study Conclusion and future work and References Conclusion and future work and References Conclusion and future work and References Supplementary Material Supplementary Material Details of the theoretical analysis on KCEMA mechanism in IIL
Algorithm overview
Dataset details
Implementation details
Visualization of dusted input images
More experimental results Details of the theoretical analysis on KCEMA mechanism in IIL Details of the theoretical analysis on KCEMA mechanism in IIL Algorithm overview Algorithm overview Dataset details Dataset details Implementation details Implementation details Visualization of dusted input images Visualization of dusted input images More experimental results More experimental results 11. Visualization of dusted input images To distill the decision boundary in an existing model, we proposed a module to dusted the input space with random Gaussian noise. By dusting the input space, we hope some samples can be relocated to the peripheral area of the learned decision boundary. Therefore, the intractable decision boundary can be manifested to some extent and distilled to the student model for knowledge retaining. The input space pollution is different with the image augmentation in the train process because of the large deviation and allowance of the polluted images to be classified to different classes besides their original labels. In fact, we hope the polluted images are prone to be classified to other category than the original category. The boundary can only be known when we know what is and what is not. The dusted input images is visualized in Fig. 10. It can be seen that the category of each image becomes vague after dusting. Authors:
(1) Qiang Nie, Hong Kong University of Science and Technology (Guangzhou);
(2) Weifu Fu, Tencent Youtu Lab;
(3) Yuhuan Lin, Tencent Youtu Lab;
(4) Jialin Li, Tencent Youtu Lab;
(5) Yifeng Zhou, Tencent Youtu Lab;
(6) Yong Liu, Tencent Youtu Lab;
(7) Qiang Nie, Hong Kong University of Science and Technology (Guangzhou);
(8) Chengjie Wang, Tencent Youtu Lab. Authors: Authors: (1) Qiang Nie, Hong Kong University of Science and Technology (Guangzhou); (2) Weifu Fu, Tencent Youtu Lab; (3) Yuhuan Lin, Tencent Youtu Lab; (4) Jialin Li, Tencent Youtu Lab; (5) Yifeng Zhou, Tencent Youtu Lab; (6) Yong Liu, Tencent Youtu Lab; (7) Qiang Nie, Hong Kong University of Science and Technology (Guangzhou); (8) Chengjie Wang, Tencent Youtu Lab. This paper is available on arxiv under CC BY-NC-ND 4.0 Deed (Attribution-Noncommercial-Noderivs 4.0 International) license. This paper is available on arxiv under CC BY-NC-ND 4.0 Deed (Attribution-Noncommercial-Noderivs 4.0 International) license. available on arxiv available on arxiv

Part of HackerNoon's growing list of open-source research papers, promoting free access to academic material.

Dusted Input Images: Visualizing Decision Boundary Distillation

About Author

Comments

TOPICS

THIS ARTICLE WAS FEATURED IN

Related Stories

Untitled Story

A Coarse-to-Fine Approach to Text-Guided 3D Localization Without Ground Truth

Data Scarcity Solution: S-CycleGAN for CT-to-Ultrasound Translation

Incremental Learning: Comparing Methods for Catastrophic Forgetting and Model Promotion

New IIL Setting: Enhancing Deployed Models with Only New Data

Optimizing SAGE Net: Achieving High Performance with Shorter Input Sequences for Online Inference

A Coarse-to-Fine Approach to Text-Guided 3D Localization Without Ground Truth

Data Scarcity Solution: S-CycleGAN for CT-to-Ultrasound Translation

Incremental Learning: Comparing Methods for Catastrophic Forgetting and Model Promotion

New IIL Setting: Enhancing Deployed Models with Only New Data

Optimizing SAGE Net: Achieving High Performance with Shorter Input Sequences for Online Inference

Light-Mode

Classic

Newspaper

Dark-Mode

Neon Noir

Minty

HN StartUps