DEVELOPMENT OF AN IOT-ENHANCED TELEPRESENCE SYSTEM BASED ON SLAM AND VR

The technology of Internet of Things (IoT) allows users to access sensing and tasking capabilities of smart devices via the Internet, which enables various automatic and efficient applications. As users usually rely on the IDs of devices on dashboards or smartphone applications to access these capabilities, users need to be familiar with the device setup and the environment in the first place. On the other hand, telepresence is the concept of providing users feelings of being exist at a remote place. While most Virtual-Reality-based (VR-based) telepresence systems provides visual, hearing, and speaking abilities to users, the ability of interacting with the environment is also necessary for an immersive telepresence system. Therefore, we propose an IoT-enhanced telepresence system based on a remote-control robot, Simultaneous Localization and Mapping (SLAM) and VR. The general steps are as follows. (1) To achieve an immersive telepresence, an omnidirectional camera is mounted on a robot, from which users can see the 360-degree view of the remote environment and control the robot with a VR device. (2) To recognize the locations of IoT devices, the position and orientation of the robot are necessary, which are obtained via aligning the ORB-SLAM2 model with a pre-generated Building Information model (BIM) model that helps indicate IoT devices. (3) Afterward, the real-time 360-degree video and control panels of IoT devices can be rendered in the VR device, where the control panels connect to an Open Geospatial Consortium (OGC) SensorThings API IoT service for sensing and tasking capabilities. As a result, the proposed system provides a more immersive telepresence experience to intuitively interact with the environment via the IoT.