Integrating depth map and IMU data for 3D reconstruction form a single image

This research developed a method for reconstructing a 3D model from a single photograph with an estimated depth map and inertial measurement unit (IMU). In photogrammertry, exterior orientation parameters (EOP) can be calculated using multiple images based on relative geometry. However, if only one image is available, this approach will not work. The proposed method uses IMU to provide the relative orientations of EOP and assumes the camera position as the origin. The estimated depth map provides initial relative distances from feature points to the camera center. Therefore, collinearity condition equations can be used for determining 3D positions of feature points from a single image. In this study, there are four main steps for 3D building reconstruction from a single image: (1) camera and boresight calibration; (2) initial depth map generation; (3) coordinates of feature points calculation and iteration; (4) 3D model reconstruction. The proposed method begins with angles calibration between camera boresight and IMU system. Boresight angles can be predicted using the result of image back projection and simultaneous recorded orientation data from IMU. Using the relative distance obtained from the initial depth map, coordinates of the extracted feature points can be determined from collinearity condition equations. The depth information is further refined by co-planarity process of coplanar feature points. Refined depth map is generated from the calculated feature points coordinates and used to compute the new coordinates. The iteration stops when the system converges or the number of the iteration reaches the pre-defined maximum. The final coordinates of feature points are then used to reconstruct the 3D model.