Deep learning-based violin bowing action recognition

Shih Wei Sun, Bao Yun Liu, Pao Chi Chang

Research output: Contribution to journalArticlepeer-review

3 Scopus citations

Abstract

We propose a violin bowing action recognition system that can accurately recognize distinct bowing actions in classical violin performance. This system can recognize bowing actions by analyzing signals from a depth camera and from inertial sensors that are worn by a violinist. The contribution of this study is threefold: (1) a dataset comprising violin bowing actions was constructed from data captured by a depth camera and multiple inertial sensors; (2) data augmentation was achieved for depth-frame data through rotation in three-dimensional world coordinates and for inertial sensing data through yaw, pitch, and roll angle transformations; and, (3) bowing action classifiers were trained using different modalities, to compensate for the strengths and weaknesses of each modality, based on deep learning methods with a decision-level fusion process. In experiments, large external motions and subtle local motions produced from violin bow manipulations were both accurately recognized by the proposed system (average accuracy > 80%).

Original languageEnglish
Article number5732
Pages (from-to)1-17
Number of pages17
JournalSensors (Switzerland)
Volume20
Issue number20
DOIs
StatePublished - 2 Oct 2020

Keywords

  • Action recognition
  • Decision level fusion
  • Deep learning applications
  • Depth camera
  • Human perceptual cognition
  • Inertial sensor
  • Violin bowing actions

Fingerprint

Dive into the research topics of 'Deep learning-based violin bowing action recognition'. Together they form a unique fingerprint.

Cite this