以人工智慧實踐二胡演奏行為暨音樂風格分析(1/2)

Erhu is a two-string fiddle musical instrument. Erhu is also known as Chineseviolin or Chinese two-stringed fiddle. In the ancient time, Erhu was used as anaccompaniment instrument in traditional Chinese opera. Erhu was made of redsandalwood or aged red wood, with python skin. With two steel strings and abamboo bow, Erhu is able to cover three and half octaves, from D4 up to A7.Learning Erhu is challenge. Usually, experiences are passed to students from aninstructor verbally. It is almost impossible to have automatic technologies forlearning Erhu. In line with the recent development of Artificial Intelligent technologies, the principal investigator of this project consulted professional Erhuresearchers, and invited professors in different technology domains. Allprofessors who join this project play Erhu and understand how computertechniques. The group aims to investigate Erhu music style, especially in thedifferences of tone characteristics used in different Erhu music. Analysis will beproceeded by using video and audio inputs, with state-of-the-art Deep Learningtechnologies as the means of qualitative tools. Another goal is to design visualeffects, as the consequence of Erhu performance visualization. A few publicperformances will be organized. These performances will be equipped with artdesigns and visualized multimedia outputs as cross-disciplinary creative works.This project is entitled “Erhu Performance and Music Style Analysis usingArtificial Intelligence.” There are four sub-projects below:1.Analysis of Erhu Performance Techniques and Music Style with ToneCharacteristics: Erhu performance techniques and how tone characteristicsimpact different music style will be investigated. Quantitative analysis of Erhusound and a collection of professional Erhu performance will be used as thetraining data of Deep learning models.2.Video Analysis of Erhu Performance Behavior: Using one RGB and anotherRGB-D camera, captured video data include the top body and detailed fingermovements of an Erhu player. A few Deep Learning modules will be used forErhu player training, with mistakes visualized. With the help of audio analysis,ornaments (or embellishments) will be captured and analyzed.3.Audio Signal Analysis of Erhu Music and MIDI Development: Music ornaments(and melody) play an important role in music style. Signal of tone changes will beused as the base for style classification. Music generation will follow particularstyles. The generated music relies on Erhu MIDI, which will be developed.4.Visualization of Erhu Performance and Music Style: Visualization of commonmistakes made by Erhu beginners will help the Erhu training system. Thevisualized effects can be further enhanced with art design, which reveals themovements of player’s finger and body signature. The visualization effect canfurther include the feeling of a particular music style, feeling of player, andemotion changes of audiences.Several public performances activities will be organized. These activities mayinclude Erhu solo performance with accompaniment, multiple Erhu players with asmall accompaniment team, or performance with art designs. Illustration oftechnologies will be included in these performances.