Auto accessory segmentation and interactive try-on system

Yi Xuan Zeng, Yu Hang Kuo, Hsu Yung Cheng

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review


The convenience and diversity of online shopping makes many consumers willing to buy apparel or accessories on the web. In order to make products more attractive to users, many virtual try-on systems are developed for e-commerce applications. This paper proposes an interactive virtual try-on system combined with automatic accessory segmentation. Our system automatically retrieves the hat from images and store them in the try-on system to provide users with subsequent selection. When a user selects the hat that he or she wants to try on, the hat is placed on the proper position of the user in the image. In the stage of accessories segmentation, we perform background elimination and super-pixel segmentation. According to the color information on the hat image, the feature vector generated by the color histogram is used to select super-pixels that belong to the accessories. In the stage of try-on system, we use Kinect, which provides skeleton information, to track the user’s face and gestures. When a user selects the hat, the proposed system reads the corresponding hat information and places the hat in the appropriate location based on the results of the face tracking. The proposed try-on system can reach 30 fps real-time speed in a personal computer.

Original languageEnglish
Title of host publicationMultiMedia Modeling - 24th International Conference, MMM 2018, Proceedings
EditorsAhmed Elgammal, Thanarat H. Chalidabhongse, Supavadee Aramvith, Yo-Sung Ho, Klaus Schoeffmann, Chong Wah Ngo, Noel E. O'Connor, Moncef Gabbouj
PublisherSpringer Verlag
Number of pages5
ISBN (Print)9783319735993
StatePublished - 2018
Event24th International Conference on MultiMedia Modeling, MMM 2018 - Bangkok, Thailand
Duration: 5 Feb 20187 Feb 2018

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume10705 LNCS
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349


Conference24th International Conference on MultiMedia Modeling, MMM 2018


  • Image segmentation
  • Kinect
  • Try-on system


Dive into the research topics of 'Auto accessory segmentation and interactive try-on system'. Together they form a unique fingerprint.

Cite this