Content-aware video analysis to guide visually impaired walking on the street

Ervin Yohannes, Timothy K. Shih, Chih Yang Lin

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

2 Scopus citations

Abstract

Although many researchers have developed systems or tools to assist blind and visually impaired people, they continue to face many obstacles in daily life—especially in outdoor environments. When people with visual impairments walk outdoors, they must be informed of objects in their surroundings. However, it is challenging to develop a system that can handle related tasks. In recent years, deep learning has enabled the development of many architectures with more accurate results than machine learning. One popular model for instance segmentation is Mask-RCNN, which can do segmentation and rapidly recognize objects. We use Mask-RCNN to develop a context-aware video that can help blind and visually impaired people recognize objects in their surroundings. Moreover, we provide the distance between the subject and object, and the object’s relative speed and direction using Mask-RCNN outputs. The results of our content-aware video include the name of the object, class object score, the distance between the person and the object, speed of the object, and object direction.

Original languageEnglish
Title of host publicationAdvances in Visual Informatics - 6th International Visual Informatics Conference, IVIC 2019, Proceedings
EditorsHalimah Badioze Zaman, Nazlena Mohamad Ali, Mohammad Nazir Ahmad, Alan F. Smeaton, Timothy K. Shih, Sergio Velastin, Tada Terutoshi
PublisherSpringer
Pages3-13
Number of pages11
ISBN (Print)9783030340315
DOIs
StatePublished - 2019
Event6th International Conference on Advances in Visual Informatics, IVIC 2019 - Bangi, Malaysia
Duration: 19 Nov 201921 Nov 2019

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume11870 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

Conference6th International Conference on Advances in Visual Informatics, IVIC 2019
Country/TerritoryMalaysia
CityBangi
Period19/11/1921/11/19

Keywords

  • Assistive technology
  • Content-aware
  • Direction
  • Distance
  • Mask-RCNN
  • Speed
  • Visually impaired

Fingerprint

Dive into the research topics of 'Content-aware video analysis to guide visually impaired walking on the street'. Together they form a unique fingerprint.

Cite this