Loose Hand Gesture Recognition Using CNN

Chen Ming Chang, Din Chang Tseng

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

4 Scopus citations


A precise hand gesture recognition (HGR) system is an important facility for human-computer interaction (HCI). In this paper, we propose a multi-resolution convolutional neural network (CNN) to recognize the loose hand gesture, where loose means that the gestures can be more varied on the bending degrees of fingers, on the direction of palm, and on the bending angles of wrist.The proposed loose hand gesture recognition (LHGR) system learn the low-level features from both color and depth images and then concatenate the low-level features to learn the RGBD (RGB color and Depth) high-level features. The advantage is that it not only suppresses the problem of the inaccurate alignment pixels between color images and deep images, but also reduce the parameters of the CNN model. In addition, we use multi-resolution features to classify the hand gestures; therefore, the proposed model has stronger ability for smaller, farther, and blurrier images. In the training stage, we trained the proposed CNN model using various loose hand gestures to make the CNN more robust. In the experiments, we compared the proposed CNN model in several different architectures; the mAP (mean average precision) is highly to 0.9973. The proposed method has reliability in the scaling and rotation of hand gestures.

Original languageEnglish
Title of host publicationAdvances in 3D Image and Graphics Representation, Analysis, Computing and Information Technology - Methods and Algorithms, Proceedings of IC3DIT 2019
EditorsRoumen Kountchev, Srikanta Patnaik, Junsheng Shi, Margarita N. Favorskaya
Number of pages10
ISBN (Print)9789811538629
StatePublished - 2020
EventAnnual International Conference on 3D Imaging Technology, IC3DIT 2019 - Kunming, China
Duration: 15 Aug 201918 Aug 2019

Publication series

NameSmart Innovation, Systems and Technologies
ISSN (Print)2190-3018
ISSN (Electronic)2190-3026


ConferenceAnnual International Conference on 3D Imaging Technology, IC3DIT 2019


  • Convolutional neural network
  • Depth image
  • Hand gesture recognition
  • Loose gesture
  • RGB image


Dive into the research topics of 'Loose Hand Gesture Recognition Using CNN'. Together they form a unique fingerprint.

Cite this