Content-aware video analysis to guide visually impaired walking on the street

Although many researchers have developed systems or tools to assist blind and visually impaired people, they continue to face many obstacles in daily life—especially in outdoor environments. When people with visual impairments walk outdoors, they must be informed of objects in their surroundings. However, it is challenging to develop a system that can handle related tasks. In recent years, deep learning has enabled the development of many architectures with more accurate results than machine learning. One popular model for instance segmentation is Mask-RCNN, which can do segmentation and rapidly recognize objects. We use Mask-RCNN to develop a context-aware video that can help blind and visually impaired people recognize objects in their surroundings. Moreover, we provide the distance between the subject and object, and the object’s relative speed and direction using Mask-RCNN outputs. The results of our content-aware video include the name of the object, class object score, the distance between the person and the object, speed of the object, and object direction.