Content-aware video analysis to guide visually impaired walking on the street

Ervin Yohannes; Timothy K. Shih; Chih Yang Lin

doi:10.1007/978-3-030-34032-2_1

Content-aware video analysis to guide visually impaired walking on the street

Ervin Yohannes, Timothy K. Shih, Chih Yang Lin

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

5 Scopus citations

Abstract

Although many researchers have developed systems or tools to assist blind and visually impaired people, they continue to face many obstacles in daily life—especially in outdoor environments. When people with visual impairments walk outdoors, they must be informed of objects in their surroundings. However, it is challenging to develop a system that can handle related tasks. In recent years, deep learning has enabled the development of many architectures with more accurate results than machine learning. One popular model for instance segmentation is Mask-RCNN, which can do segmentation and rapidly recognize objects. We use Mask-RCNN to develop a context-aware video that can help blind and visually impaired people recognize objects in their surroundings. Moreover, we provide the distance between the subject and object, and the object’s relative speed and direction using Mask-RCNN outputs. The results of our content-aware video include the name of the object, class object score, the distance between the person and the object, speed of the object, and object direction.

Original language	English
Title of host publication	Advances in Visual Informatics - 6th International Visual Informatics Conference, IVIC 2019, Proceedings
Editors	Halimah Badioze Zaman, Nazlena Mohamad Ali, Mohammad Nazir Ahmad, Alan F. Smeaton, Timothy K. Shih, Sergio Velastin, Tada Terutoshi
Publisher	Springer
Pages	3-13
Number of pages	11
ISBN (Print)	9783030340315
DOIs	https://doi.org/10.1007/978-3-030-34032-2_1
State	Published - 2019
Event	6th International Conference on Advances in Visual Informatics, IVIC 2019 - Bangi, Malaysia Duration: 19 Nov 2019 → 21 Nov 2019

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	11870 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Conference

Conference	6th International Conference on Advances in Visual Informatics, IVIC 2019
Country/Territory	Malaysia
City	Bangi
Period	19/11/19 → 21/11/19

Keywords

Assistive technology
Content-aware
Direction
Distance
Mask-RCNN
Speed
Visually impaired

Access to Document

10.1007/978-3-030-34032-2_1

Cite this

Yohannes, E., Shih, T. K., & Lin, C. Y. (2019). Content-aware video analysis to guide visually impaired walking on the street. In H. Badioze Zaman, N. Mohamad Ali, M. N. Ahmad, A. F. Smeaton, T. K. Shih, S. Velastin, & T. Terutoshi (Eds.), Advances in Visual Informatics - 6th International Visual Informatics Conference, IVIC 2019, Proceedings (pp. 3-13). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11870 LNCS). Springer. https://doi.org/10.1007/978-3-030-34032-2_1

Yohannes, Ervin ; Shih, Timothy K. ; Lin, Chih Yang. / Content-aware video analysis to guide visually impaired walking on the street. Advances in Visual Informatics - 6th International Visual Informatics Conference, IVIC 2019, Proceedings. editor / Halimah Badioze Zaman ; Nazlena Mohamad Ali ; Mohammad Nazir Ahmad ; Alan F. Smeaton ; Timothy K. Shih ; Sergio Velastin ; Tada Terutoshi. Springer, 2019. pp. 3-13 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{fdb66875498b46abb217d6b27e100bb5,

title = "Content-aware video analysis to guide visually impaired walking on the street",

abstract = "Although many researchers have developed systems or tools to assist blind and visually impaired people, they continue to face many obstacles in daily life—especially in outdoor environments. When people with visual impairments walk outdoors, they must be informed of objects in their surroundings. However, it is challenging to develop a system that can handle related tasks. In recent years, deep learning has enabled the development of many architectures with more accurate results than machine learning. One popular model for instance segmentation is Mask-RCNN, which can do segmentation and rapidly recognize objects. We use Mask-RCNN to develop a context-aware video that can help blind and visually impaired people recognize objects in their surroundings. Moreover, we provide the distance between the subject and object, and the object{\textquoteright}s relative speed and direction using Mask-RCNN outputs. The results of our content-aware video include the name of the object, class object score, the distance between the person and the object, speed of the object, and object direction.",

keywords = "Assistive technology, Content-aware, Direction, Distance, Mask-RCNN, Speed, Visually impaired",

author = "Ervin Yohannes and Shih, {Timothy K.} and Lin, {Chih Yang}",

note = "Publisher Copyright: {\textcopyright} Springer Nature Switzerland AG 2019.; 6th International Conference on Advances in Visual Informatics, IVIC 2019 ; Conference date: 19-11-2019 Through 21-11-2019",

year = "2019",

doi = "10.1007/978-3-030-34032-2_1",

language = "???core.languages.en_GB???",

isbn = "9783030340315",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer",

pages = "3--13",

editor = "{Badioze Zaman}, Halimah and {Mohamad Ali}, Nazlena and Ahmad, {Mohammad Nazir} and Smeaton, {Alan F.} and Shih, {Timothy K.} and Sergio Velastin and Tada Terutoshi",

booktitle = "Advances in Visual Informatics - 6th International Visual Informatics Conference, IVIC 2019, Proceedings",

}

Yohannes, E, Shih, TK & Lin, CY 2019, Content-aware video analysis to guide visually impaired walking on the street. in H Badioze Zaman, N Mohamad Ali, MN Ahmad, AF Smeaton, TK Shih, S Velastin & T Terutoshi (eds), Advances in Visual Informatics - 6th International Visual Informatics Conference, IVIC 2019, Proceedings. Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 11870 LNCS, Springer, pp. 3-13, 6th International Conference on Advances in Visual Informatics, IVIC 2019, Bangi, Malaysia, 19/11/19. https://doi.org/10.1007/978-3-030-34032-2_1

Content-aware video analysis to guide visually impaired walking on the street. / Yohannes, Ervin; Shih, Timothy K.; Lin, Chih Yang.
Advances in Visual Informatics - 6th International Visual Informatics Conference, IVIC 2019, Proceedings. ed. / Halimah Badioze Zaman; Nazlena Mohamad Ali; Mohammad Nazir Ahmad; Alan F. Smeaton; Timothy K. Shih; Sergio Velastin; Tada Terutoshi. Springer, 2019. p. 3-13 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 11870 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Content-aware video analysis to guide visually impaired walking on the street

AU - Yohannes, Ervin

AU - Shih, Timothy K.

AU - Lin, Chih Yang

N1 - Publisher Copyright: © Springer Nature Switzerland AG 2019.

PY - 2019

Y1 - 2019

N2 - Although many researchers have developed systems or tools to assist blind and visually impaired people, they continue to face many obstacles in daily life—especially in outdoor environments. When people with visual impairments walk outdoors, they must be informed of objects in their surroundings. However, it is challenging to develop a system that can handle related tasks. In recent years, deep learning has enabled the development of many architectures with more accurate results than machine learning. One popular model for instance segmentation is Mask-RCNN, which can do segmentation and rapidly recognize objects. We use Mask-RCNN to develop a context-aware video that can help blind and visually impaired people recognize objects in their surroundings. Moreover, we provide the distance between the subject and object, and the object’s relative speed and direction using Mask-RCNN outputs. The results of our content-aware video include the name of the object, class object score, the distance between the person and the object, speed of the object, and object direction.

AB - Although many researchers have developed systems or tools to assist blind and visually impaired people, they continue to face many obstacles in daily life—especially in outdoor environments. When people with visual impairments walk outdoors, they must be informed of objects in their surroundings. However, it is challenging to develop a system that can handle related tasks. In recent years, deep learning has enabled the development of many architectures with more accurate results than machine learning. One popular model for instance segmentation is Mask-RCNN, which can do segmentation and rapidly recognize objects. We use Mask-RCNN to develop a context-aware video that can help blind and visually impaired people recognize objects in their surroundings. Moreover, we provide the distance between the subject and object, and the object’s relative speed and direction using Mask-RCNN outputs. The results of our content-aware video include the name of the object, class object score, the distance between the person and the object, speed of the object, and object direction.

KW - Assistive technology

KW - Content-aware

KW - Direction

KW - Distance

KW - Mask-RCNN

KW - Speed

KW - Visually impaired

UR - http://www.scopus.com/inward/record.url?scp=85077839398&partnerID=8YFLogxK

U2 - 10.1007/978-3-030-34032-2_1

DO - 10.1007/978-3-030-34032-2_1

M3 - 會議論文篇章

AN - SCOPUS:85077839398

SN - 9783030340315

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 3

EP - 13

BT - Advances in Visual Informatics - 6th International Visual Informatics Conference, IVIC 2019, Proceedings

A2 - Badioze Zaman, Halimah

A2 - Mohamad Ali, Nazlena

A2 - Ahmad, Mohammad Nazir

A2 - Smeaton, Alan F.

A2 - Shih, Timothy K.

A2 - Velastin, Sergio

A2 - Terutoshi, Tada

PB - Springer

T2 - 6th International Conference on Advances in Visual Informatics, IVIC 2019

Y2 - 19 November 2019 through 21 November 2019

ER -

Yohannes E, Shih TK , Lin CY. Content-aware video analysis to guide visually impaired walking on the street. In Badioze Zaman H, Mohamad Ali N, Ahmad MN, Smeaton AF, Shih TK, Velastin S, Terutoshi T, editors, Advances in Visual Informatics - 6th International Visual Informatics Conference, IVIC 2019, Proceedings. Springer. 2019. p. 3-13. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-030-34032-2_1

Content-aware video analysis to guide visually impaired walking on the street

Abstract

Publication series

Conference

Keywords

Access to Document

Fingerprint

Cite this