A Novel Self-Knowledge Distillation Approach with Siamese Representation Learning for Action Recognition

Duc Quang Vu, Thi Thu Trang Phung, Jia Ching Wang

研究成果: 書貢獻/報告類型會議論文篇章同行評審

摘要

Knowledge distillation is an effective transfer of knowledge from a heavy network (teacher) to a small network (student) to boost students' performance. Self-knowledge dis-tillation, the special case of knowledge distillation, has been proposed to remove the large teacher network training process while preserving the student's performance. This paper intro-duces a novel Self-knowledge distillation approach via Siamese representation learning, which minimizes the difference between two representation vectors of the two different views from a given sample. Our proposed method, SKD-SRL, utilizes both soft label distillation and the similarity of representation vectors. Therefore, SKD-SRL can generate more consistent predictions and representations in various views of the same data point. Our benchmark has been evaluated on various standard datasets. The experimental results have shown that SKD-SRL significantly improves the accuracy compared to existing supervised learning and knowledge distillation methods regardless of the networks.

原文???core.languages.en_GB???
主出版物標題2021 International Conference on Visual Communications and Image Processing, VCIP 2021 - Proceedings
發行者Institute of Electrical and Electronics Engineers Inc.
ISBN(電子)9781728185514
DOIs
出版狀態已出版 - 2021
事件2021 International Conference on Visual Communications and Image Processing, VCIP 2021 - Munich, Germany
持續時間: 5 12月 20218 12月 2021

出版系列

名字2021 International Conference on Visual Communications and Image Processing, VCIP 2021 - Proceedings

???event.eventtypes.event.conference???

???event.eventtypes.event.conference???2021 International Conference on Visual Communications and Image Processing, VCIP 2021
國家/地區Germany
城市Munich
期間5/12/218/12/21

指紋

深入研究「A Novel Self-Knowledge Distillation Approach with Siamese Representation Learning for Action Recognition」主題。共同形成了獨特的指紋。

引用此