Knowledge Distillation with Adversarial Samples Supporting Decision Boundary
- Title
- Knowledge Distillation with Adversarial Samples Supporting Decision Boundary
- Author
- 이민식
- Issue Date
- 2019-01
- Publisher
- AAAI
- Citation
- Proceedings of the AAAI Conference on Artificial Intelligence, v. 33, Page. 3771-3778
- Abstract
- Many recent works on knowledge distillation have provided ways to transfer the knowledge of a trained network for improving the learning process of a new one, but finding a good technique for knowledge distillation is still an open problem. In this paper, we provide a new perspective based on a decision boundary, which is one of the most important component of a classifier. The generalization performance of a classifier is closely related to the adequacy of its decision boundary, so a good classifier bears a good decision boundary. Therefore, transferring information closely related to the decision boundary can be a good attempt for knowledge distillation. To realize this goal, we utilize an adversarial attack to discover samples supporting a decision boundary. Based on this idea, to transfer more accurate information about the decision boundary, the proposed algorithm trains a student classifier based on the adversarial samples supporting the decision boundary. Experiments show that the proposed method indeed improves knowledge distillation and achieves the state-of-the-arts performance.
- URI
- https://www.aaai.org/ojs/index.php/AAAI/article/view/4263https://repository.hanyang.ac.kr/handle/20.500.11754/112144
- DOI
- 10.1609/aaai.v33i01.33013771
- Appears in Collections:
- COLLEGE OF ENGINEERING SCIENCES[E](공학대학) > ELECTRICAL ENGINEERING(전자공학부) > Articles
- Files in This Item:
There are no files associated with this item.
- Export
- RIS (EndNote)
- XLS (Excel)
- XML