HandKey: An Efficient Hand Typing Recognition using CNN for Virtual Keyboard

Avirmed Enkhbat, Timothy K. Shih, Tipajin Thaipisutikul, Noorkholis Luthfil Hakim, Wisnu Aditya

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

Abstract

This paper proposes an efficient framework that recognizes hand typing motions and gestures for making a virtual keyboard by using a single RGB camera. There are several works related to virtual keyboard in the Human-computer interaction (HCI) area. Most of them use hand pose estimation, hand shape and external equipment (depth sensor, leap motion, control glove, touch screen etc.). Whereas, our framework does not require additional equipment or prior experience from users, it works like a regular typing action in the air which is similar to typing on a real QWERTY keyboard. It uses convolutional neural networks (CNN) to classify 2 hand typing gestures (touch and non-touch). Also, we train 11 gestures which are non-touch and touching for each 10 fingers of two hands gestures. Proposed CNN model achieves a 99.2% classification accuracy for the 2 gestures case and a 91% classification accuracy for the 11 gestures case.

Original languageEnglish
Title of host publicationInCIT 2020 - 5th International Conference on Information Technology
PublisherInstitute of Electrical and Electronics Engineers Inc.
Pages315-319
Number of pages5
ISBN (Electronic)9781728166940
DOIs
StatePublished - 21 Oct 2020
Event5th International Conference on Information Technology, InCIT 2020 - Chon Buri, Thailand
Duration: 21 Oct 202022 Oct 2020

Publication series

NameInCIT 2020 - 5th International Conference on Information Technology

Conference

Conference5th International Conference on Information Technology, InCIT 2020
Country/TerritoryThailand
CityChon Buri
Period21/10/2022/10/20

Keywords

  • convolutional neural network
  • hand typing gesture recognition
  • human-computer interaction
  • motion history image
  • virtual keyboard

Fingerprint

Dive into the research topics of 'HandKey: An Efficient Hand Typing Recognition using CNN for Virtual Keyboard'. Together they form a unique fingerprint.

Cite this