A Skeleton-based Dynamic Hand Gesture Recognition for Home Appliance Control System

Tsung Han Tsai, Yi Jhen Luo, Wei Chung Wan

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

4 Scopus citations

Abstract

In recent years, advances in 3D sensors have dramatically promoted the development of dynamic hand gesture recognition research. On the other side, the task of hand pose estimation has seen significant progress due to the powerful feature extraction capabilities based on Convolutional Neural Networks (CNNs). In this paper, we present a lightweight CNNs method on hand gesture recognition for home appliance control system. We propose a two-stage CNN model to facilitate it. At the first stage, we utilize DetNet to detect the hand and generate 3D hand skeleton locations. At the second stage, a skeleton-based dynamic hand gesture recognition model is developed. We have 99.4% accuracy by the trained CNN model with the testing dataset. Besides, we implement this system on the Nvidia Jetson AGX Xavier to control the on/off of the fan and the light.

Original languageEnglish
Title of host publicationIEEE International Symposium on Circuits and Systems, ISCAS 2022
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages3265-3268
Number of pages4
ISBN (Electronic)9781665484855
DOIs
StatePublished - 2022
Event2022 IEEE International Symposium on Circuits and Systems, ISCAS 2022 - Austin, United States
Duration: 27 May 20221 Jun 2022

Publication series

NameProceedings - IEEE International Symposium on Circuits and Systems
Volume2022-May
ISSN (Print)0271-4310

Conference

Conference2022 IEEE International Symposium on Circuits and Systems, ISCAS 2022
Country/TerritoryUnited States
CityAustin
Period27/05/221/06/22

Keywords

  • computer vision
  • convolutional neural networks
  • deep learning
  • dynamic hand gesture recognition
  • home appliance application
  • skeleton

Fingerprint

Dive into the research topics of 'A Skeleton-based Dynamic Hand Gesture Recognition for Home Appliance Control System'. Together they form a unique fingerprint.

Cite this