Deep Recognition of Facial Expressions in Movies

Consumer feedback is often used for various purposes in many fields. However, traditional paper questionnaires or surveys cannot fully meet the demands for accurately understanding consumers’ feelings. Consumers often convey their feelings through their facial expressions, whether consciously or unconsciously. Understanding these feelings can provide very direct and useful feedbacks. Yet, humans may miss those subtle changes because the micro expression is too brief to be captured. Therefore, in this paper, we proposed a deep learning based recognition approach of facial micro expressions, in which more realistic emotional feedback of users can be extracted. To achieve this goal, we integrated several approaches including: (1) using trained face detection model to capture face image from input; (2) training a high accurate 468-point landmark detection model with multiple face dataset. Based on the FACS (Facial Action Coding System) table, we categorized these landmarks into 13 groups of facial regions. These regions with specific emotion labels are used as our target units of AU (Action Unit) detection; (3) training CNN model to detect and analyze AUs from facial landmark data; (4) implying FACS to evaluate the facial expressions and emotions; and (5) using a straightforward GUI plotter to show the digitized emotions. The experiment results show that not only the primary emotion but also the secondary emotion of users in movies can be detected and evaluated successfully. Therefore, our system has a great potential for obtaining users’ feedbacks in a more accurate and comprehensive manner.