A robust passage retrieval algorithm for video question answering

Yu Chieh Wu, Jie Chi Yang

Research output: Contribution to journalArticlepeer-review

20 Scopus citations


In this paper, we present a robust passage retrieval algorithm to extend the conventional text question answering (Q/A) to videos. Users interact with our videoQ/A system through natural language queries, while the top-ranked passage fragments with associated video clips are returned as answers. We compare our method with five of the high-performance ranking algorithms that are portable to different languages and domains. The experiments were evaluated with 75.3 h of Chinese videos and 253 questions. The experimental results showed that our method outperformed the second best retrieval model (language models) in relatively 1.43% in mean reciprocal rank (MRR) score and 11.36% when employing a Chinese word segmentation tool. By adopting the initial retrieval results from the retrieval models, our method yields an improvement of at least 5.94% improvement in MRR score. This makes it very attractive for the Asia-like languages since the use of a well-developed word tokenizer is unnecessary.

Original languageEnglish
Article number4633656
Pages (from-to)1411-1421
Number of pages11
JournalIEEE Transactions on Circuits and Systems for Video Technology
Issue number10
StatePublished - Oct 2008


  • Multimedia retrieval
  • Question answering (Q/A)
  • Video question answering (videoQ/A)


Dive into the research topics of 'A robust passage retrieval algorithm for video question answering'. Together they form a unique fingerprint.

Cite this