A Video Analytic In-Class Student Concentration Monitoring System

Mu Chun Su, Chun Ting Cheng, Ming Ching Chang, Yi Zeng Hsieh

研究成果: 雜誌貢獻期刊論文同行評審

15 引文 斯高帕斯(Scopus)


Automatic learning feedback monitoring and analysis are becoming essential in modern education. We present a video analytic system capable of monitoring in-class student's learning behaviors and providing feedback to the instructor. It is a common practice nowadays for students to take electronic notes or browse online using laptops and cellphones in class. However the use of technology can also impact student concentration and affect learning behaviors, which can seriously hinder their learning progress if not controlled properly. In this pioneering study, we propose a non-intrusive deep-learning based computer vision system to monitor student concentration by extracting and inferring high-level visual behavior cues, including their facial expressions, gestures and activities. Our system can automatically assist instructors with situational awareness in real time. We assume only RGB color images as input and runable system on edge devices for easy deployment. We propose two video analytic components for student behavior analysis: (1) The facial analysis component operates based on Dlib face detection and facial landmark tracking to localize each student and analyze their face orientations, eye blinking, gazes, and facial expressions. (2) The activity detection and recognition component operates based on OpenPose and COCO object detection can identify eight types of in-class gestures and behaviors including raising-hand, typing, phone-answering, crooked-head, desk napping, etc. Experiments are performed on a newly collected real-world In-Class Student Activity Dataset (ICSAD), where we achieved nearly 80% activity detection rate. Our system is view-independent in handling facial and pose orientations with average angular error < 10°. The source code of this work is at: https://github.com/YiZengHsieh/ICSAD.

頁(從 - 到)294-304
期刊IEEE Transactions on Consumer Electronics
出版狀態已出版 - 1 11月 2021


深入研究「A Video Analytic In-Class Student Concentration Monitoring System」主題。共同形成了獨特的指紋。