2021 IEEE International Conference on Acoustics, Speech and Signal Processing

6-11 June 2021 • Toronto, Ontario, Canada

Extracting Knowledge from Information

2021 IEEE International Conference on Acoustics, Speech and Signal Processing

6-11 June 2021 • Toronto, Ontario, Canada

Extracting Knowledge from Information

Technical Program

Paper Detail

Paper IDIVMSP-28.1
Paper Title SEMANTIC IMAGE SYNTHESIS FROM INACCURATE AND COARSE MASKS
Authors Kai Katsumata, Hideki Nakayama, University of Tokyo, Japan
SessionIVMSP-28: Image Synthesis
LocationGather.Town
Session Time:Friday, 11 June, 11:30 - 12:15
Presentation Time:Friday, 11 June, 11:30 - 12:15
Presentation Poster
Topic Image, Video, and Multidimensional Signal Processing: [IVTEC] Image & Video Processing Techniques
IEEE Xplore Open Preview  Click here to view in IEEE Xplore
Virtual Presentation  Click here to watch in the Virtual Conference
Abstract Semantic image synthesis is an image-to-image translation problem where the goal is to learn mapping from semantic segmentation masks to corresponding photorealistic images. However, conventional semantic image synthesis methods require numerous pairs of correct semantic masks and real images, and collecting these pairs is not always possible. To address this issue, we propose a smoothing method, which we call local label smoothing (LLS), that incorporates label smoothing per small patch of an input mask to learn mapping from masks to images even when semantic masks are inaccurate. Furthermore, we also propose an extended method for coarse masks. We demonstrate the advantage of the proposed methods over existing methods to deal with noisy masks on several datasets.