Quality analysis of object categorization benchmarks: Corel, caltech, and IAPR datasets

Chih Fong Tsai, Wei Chao Lin, Chihli Hung

Research output: Contribution to journalReview articlepeer-review


Image retrieval aims at searching similar semantic contents of images to the users' queries. However, the major challenge of image retrieval is to make computers understand high-level concepts of images, e.g. objects. In other words, automatic image annotation by object recognition and categorization can achieve semantic-based image retrieval. However, object categorization is a very important and challenging problem in computer vision. To evaluate some proposed object categorization algorithm, a pilot experiment is usually conducted based on a chosen small scale dataset in order to quickly obtain some useful results for later large scale experiments. Although there are a number of object categorization benchmarks, it is time consuming to consider all of them for the pilot experiment. This patent paper analyzes three wellknown benchmarks, which are Corel, IAPR, and Caltech, and tries to identify the levels of easiness and difficulty for image annotation. In addition, relevant patents are also reviewed in this manuscript. In the experiments, local region-based image features were extracted from these datasets and the Euclidean distances between images were analyzed in order to understand the distance distribution of images in the feature space. Moreover, image annotation accuracy using SVM and k-NN classifiers were examined. The experimental results showed that Caltech is the easiest dataset, which allows classifiers to provide the highest rate of annotation accuracy. On the other hand, IAPR is the toughest dataset and Corel stands at the middle position.

Original languageEnglish
Pages (from-to)108-113
Number of pages6
JournalRecent Patents on Computer Science
Issue number2
StatePublished - May 2011


  • Benchmarks
  • Caltech
  • Corel
  • IAPR
  • Image annotation
  • Image retrieval
  • Object categorization


Dive into the research topics of 'Quality analysis of object categorization benchmarks: Corel, caltech, and IAPR datasets'. Together they form a unique fingerprint.

Cite this