Watching online videos is a major leisure activity among Internet users. The largest video website, YouTube, stores billions of videos on its servers. Thus, previous studies have applied automatic video categorization methods to enable users to find videos corresponding to their needs; however, emotion has not been a factor considered in these classification methods. Therefore, this study classified YouTube videos into six emotion categories (i.e., happiness, anger, disgust, fear, sadness, and surprise). Through unsupervised and supervised learning methods, this study first categorized videos according to emotion. An ensemble model was subsequently applied to integrate the classification results of both methods. The experimental results confirm that the proposed method effectively facilitates the classification of YouTube videos into suitable emotion categories.