Stereo-based 3D space handwriting recognition

Recently, based on the improvements of hardware performance and the popularity of internet, big data analysis and artificial intelligence were successfully applied in a wide range of applications. Similarly, computer vision technology also benefited from the powerful performance of hardware and artificial intelligence, so that the computer vision technology could solve problem more efficiently and accurately and improve the development of automation. In this thesis, we aim at measuring the distance between the finger and camera, and tracking the finger to fulfill a stereo vision-based hand-writing recognition system in three-dimensional (3D) space. Traditionally, the researchers usually applied infrared sensors to recognize human's hands. However, the infrared sensor solution still be challenged in hand tracking algorithm under widely varying lighting, distance limitation, and the outdoor condition. As mentioned above, this thesis attempts to generate the depth information based on stereo vision for improving the finger tracking. Through the depth information, we determine and track the fingers step by step. Also, tracking target would be excluded from other objects and background. In this thesis, the Probability Density Function is applied to get the threshold value, which could find out the region of interest automatically instead of manually. Furthermore, the proposed system uses Particle Swarm Optimization for hand tracking. After getting the hand (palm) position in each frame, the grayscale image would be used to analyze the fingers. Finally, the multilayer perceptron is used to train the MNIST dataset for hand-writing character validation. The experimental results demonstrate that the proposed system could recognize hand-writing digits in 3D space in high accuracy without any constraints and restricted environment.