360-Degree Vi Deo Processing and Analysis for Vi Rtual Reality Applications

For virtual reality applications, head-mounted displays (HMDs) with 360 degree videos provide users immersive experiences. However, several new challenges also arise: (1) 360 degree videos with high spatial resolution and high frame rate lead to a large amount of data (4K and beyond). (2) Sphere-to-plane projections such as equirectangular projection and polyhedron projection may have inevitable geometric deformation. (3) Viewport streaming should promptly respond to head movement of viewers. State-of-the-art methods of 2D video coding (e.g., HEVC) still cannot make the compressed bitstream of 360 degree videos meet the constraint of network latency. Moreover, most existing object tracking techniques cannot be applied to current 360 degree videos with variety of projection formats. Thus, the major goals of this project consist of two parts of 360 degree video processing and analysis, and they are stated as follows: (1) Rate control of 360 degree video coding. (2) Object tracking on 360 degree videos. This project expects to increase accuracy of rate control of 360 degree video coding, smooth visual quality and buffer fullness of 360 degree video coding, and increase accuracy of object tracking. Thus, quality of viewing experiences for virtual reality applications and accuracy of 360 degree video analysis can be significantly improved.