Intelligent Guiding Glasses with Descriptive Video Service and Spoken Dialogue System for Visually Impaired

Project Details


In order to overcome the inconvenience of visually impaired people, this project proposes a novel smart wearable device for walking navigation. It is a pair of glasses combining 3D vision and microphone, which can help the visually impaired to be safe and convenient in outdoor activities. The project will take three years. In the first year, we used 3D detection and recognition to handle visual understanding of obstacles and the environment, and turn the recognized information into spoken language. Then the following year, a dialogue system based on the needs of the blind will be developed. Through man-machine dialogue in natural language, information on street views, road conditions, and walking directions and distances in front of blind people will be clarified. In the last year, the software and hardware integration of smart guide glasses will be finished to completed the development of a guide system for the visually impaired. We will invite blind people to evaluate the glasses in real environment for improving the practicability and performance of the system.
Effective start/end date1/08/2031/07/21

UN Sustainable Development Goals

In 2015, UN member states agreed to 17 global Sustainable Development Goals (SDGs) to end poverty, protect the planet and ensure prosperity for all. This project contributes towards the following SDG(s):

  • SDG 12 - Responsible Consumption and Production
  • SDG 15 - Life on Land
  • SDG 17 - Partnerships for the Goals


  • dialogue system
  • image captioning
  • deep learning
  • street recognition
  • natural language process


Explore the research topics touched on by this project. These labels are generated based on the underlying awards/grants. Together they form a unique fingerprint.