A Combined Svm/Cnn Deep Learning Architecture for Improving Coding Performance and Computational Complexity of Hevc

In the last project we have been investigating using support vector machine (SVM) techniques for High Efficiency Video Coding (HEVC) to reduce its computational complexity, in special for previously proposed halftoned HEVC intra prediction and SABPD-based inter prediction. Experimental results demonstrate that average 22% of total encoding time can be reduced for HEVC intra prediction, but with only slightly increase in bit rate (only with average 0.09% bit rate increment). The results also show that 30% of computation time can be saved for HEVC inter prediction, and the bit rate increment is less than 0.1%, compared to the original HEVC. The proposed QP-optimized SVM algorithms have been realized in HEVC reference software, and parts of the results have been presented in IEEE international conferences such as GCCE etc.The developed QP-optimized SVM algorithm has very good coding performance, with less than 0.1% bit rate increment, compared to the original HEVC. This is due to that the SVM algorithm has very high classification accuracy (98% accuracy on average) to classify CU or PU modes. In recent years the convolutional neural network (CNN)-based deep learning technique has been widely used in HEVC to improve the coding performance or reduce the computational complexity. In this three year's project we will incorporate the developed SVM method into the CNN and will study the performance of the novel combined SVM/CNN architecture. Based upon the developed SVM algorithm we first classify a video sequence (coded tree blocks, CTUs) into some subgroups, and the CTUs within each subgroup has very high similar characteristics. Each subgroup is processed by its own CNN model and as a result more accurate and better CNN models can be obtained when compared with a single CNN model that is processed for all possible CTUs that has varieties of characteristics. In the first year, we will investigate the novel SVM/CNN architecture for HEVC to improve the coding performance. The coding performance will be studied and compared with that using CNN only. In the second year we will focus on reducing the computational complexity of HEVC with the new SVM/CNN architecture. The computation efficiency will be investigated and compared with those using SVM or CNN independently. In the last year, we will integrate all investigated algorithms together and investigate both coding efficiency and computation time, and compare with those using SVM or CNN only. We will also develop and implement new QP-optimized SVM/CNN architecture into HEVC reference software.