Exploiting Style Transfer and Semantic Segmentation to Facilitate Infrared and Visible Image Fusion

Image fusion integrates different imaging sources to generate one with improved scene representation or visual perception, supporting advanced vision tasks such as object detection and semantic analysis. Fusing infrared and visible images is a widely studied subject, and the current trend is to adopt deep learning models. It is well known that training a deep fusion model often requires many labeled data. Nevertheless, existing datasets only provide images without precise annotations, affecting the fusion presentation and limiting further development. This research creates a dataset for infrared and visible image fusion with semantic segmentation information. We utilize existing image datasets specific to semantic segmentation and generate corresponding infrared images by style transferring. A labeled dataset for image fusion is formed, in which each pair of infrared and visible images is accompanied by their semantic segmentation labels. The performance of image fusion in target datasets can thus be improved.