SATD-based intra mode decision for H.264/AVC video coding

Yu Ming Lee, Yu Ting Sun, Yinyi Lin

Research output: Contribution to journalArticlepeer-review

42 Scopus citations


In this letter, a rate estimation technique using transform coefficient variance is proposed in the rate-distortion cost function for intra mode decision. The new cost function achieves better coding performance than the cost function using the p-domain model. The new cost function brings out an average speed-up factor of 13 when compared with the rate-distortion optimization (RDO) mode decision, but with considerable degradation in rate-distortion performance. To improve the coding performance, we propose a sum of the absolute transformed differences (SATD)-based mode decision algorithm, which incorporates both SATD and the variance information into the RDO mode decision method. The results show that the SATD-based mode decision scheme can achieve a significant improvement in computation (with 21% of total encoding time, or a speed-up factor of 5) with negligible peak signal-to-noise ratio loss. The proposed algorithm is also compared with two other distinct algorithms, and the results indicate that the proposed algorithm outperforms these two edge detection algorithms in coding performance, as well as time saving.

Original languageEnglish
Article number5308396
Pages (from-to)463-469
Number of pages7
JournalIEEE Transactions on Circuits and Systems for Video Technology
Issue number3
StatePublished - Mar 2010


  • H264/AVC
  • Intra mode decision
  • P-domain model
  • Rate-distortion cost function
  • Rate-distortion optimization (RDO)


Dive into the research topics of 'SATD-based intra mode decision for H.264/AVC video coding'. Together they form a unique fingerprint.

Cite this