Multi-Modal Deep Learning-Based Violin Bowing Action Recognition

Bao Yun Liu, Yi Hsin Jen, Shih Wei Sun, Li Su, Pao Chi Chang

研究成果: 書貢獻/報告類型會議論文篇章同行評審

摘要

In this paper, a deep learning-based violin action recognition is proposed. By fusing the sensing signals from depth camera modality and inertial sensor modalities, violin bowing actions can be recognized by the proposed deep learning scheme. The actions performed by a violinist are captured by a depth camera, and recorded by wearable sensors on the forearm of a violinist. In the proposed system, 3D convolution neural network (3D-CNN) and long short-term memory (LSTM) deep learning algorithms are adopted to generate the action models from depth camera modality and inertial sensor modalities. The features and models obtained from multi-modalities are used to classify different violin bowing actions. A fusion process from different modalities can achieve satisfactory recognition accuracy. In this paper, we generate a violin bowing actions dataset for the preliminary study and the system performance evaluation.

原文???core.languages.en_GB???
主出版物標題2020 IEEE International Conference on Consumer Electronics - Taiwan, ICCE-Taiwan 2020
發行者Institute of Electrical and Electronics Engineers Inc.
ISBN(電子)9781728173993
DOIs
出版狀態已出版 - 28 9月 2020
事件7th IEEE International Conference on Consumer Electronics - Taiwan, ICCE-Taiwan 2020 - Taoyuan, Taiwan
持續時間: 28 9月 202030 9月 2020

出版系列

名字2020 IEEE International Conference on Consumer Electronics - Taiwan, ICCE-Taiwan 2020

???event.eventtypes.event.conference???

???event.eventtypes.event.conference???7th IEEE International Conference on Consumer Electronics - Taiwan, ICCE-Taiwan 2020
國家/地區Taiwan
城市Taoyuan
期間28/09/2030/09/20

指紋

深入研究「Multi-Modal Deep Learning-Based Violin Bowing Action Recognition」主題。共同形成了獨特的指紋。

引用此