Self-Supervised Learning via Multi-Transformation Classification for Action Recognition

Duc Quang Vu, Ngan Le, Jia Ching Wang

研究成果: 書貢獻/報告類型會議論文篇章同行評審

1 引文 斯高帕斯(Scopus)

摘要

Self-supervised tasks have been utilized to build useful representations that can be used in downstream tasks when the annotation is unavailable. In this paper, we introduce a self-supervised video representation learning method based on the multi-transformation classification to efficiently classify human actions. Self-supervised learning on various transformations not only provides richer contextual information but also enables the visual representation more robust to the transforms. The spatio-temporal representation of the video is learned in a self-supervised manner by classifying seven different transformations i.e. rotation, clip inversion, permutation, split, join transformation, color switch, frame replacement, and noise addition. First, seven different video transformations are applied to video clips. Then the 3D convolutional neural networks are utilized to extract features for clips and these features are processed to classify the pseudo-labels. We use the learned models in pretext tasks as the pre-trained models and fine-tune them to recognize human actions in the downstream task. We have conducted the experiments on UCF101 and HMDB51 datasets together with C3D and 3D Resnet-18 as backbone networks. The experimental results have shown that our proposed framework outperformed other SOTA self-supervised action recognition approaches.

原文???core.languages.en_GB???
主出版物標題2024 IEEE International Conference on Multimedia and Expo Workshops, ICMEW 2024
發行者Institute of Electrical and Electronics Engineers Inc.
ISBN(電子)9798350379815
DOIs
出版狀態已出版 - 2024
事件2024 IEEE International Conference on Multimedia and Expo Workshops, ICMEW 2024 - Niagara Falls, Canada
持續時間: 15 7月 202419 7月 2024

出版系列

名字2024 IEEE International Conference on Multimedia and Expo Workshops, ICMEW 2024

???event.eventtypes.event.conference???

???event.eventtypes.event.conference???2024 IEEE International Conference on Multimedia and Expo Workshops, ICMEW 2024
國家/地區Canada
城市Niagara Falls
期間15/07/2419/07/24

指紋

深入研究「Self-Supervised Learning via Multi-Transformation Classification for Action Recognition」主題。共同形成了獨特的指紋。

引用此