2021 IEEE International Conference on Acoustics, Speech and Signal Processing
6-11 June 2021 • Toronto, Ontario, Canada
Extracting Knowledge from Information
| Paper ID | IVMSP-18.6 | ||
| Paper Title | Bridging Unpaired Facial Photos and Sketches by Line-drawings | ||
| Authors | Meimei Shang, Fei Gao, Xiang Li, Hangzhou Dianzi University, China; Jingjie Zhu, AiSketcher Technology Co. Ltd., China; Lingna Dai, Hangzhou Dianzi University, China | ||
| Session | IVMSP-18: Faces in Images & Videos | ||
| Location | Gather.Town | ||
| Session Time: | Wednesday, 09 June, 16:30 - 17:15 | ||
| Presentation Time: | Wednesday, 09 June, 16:30 - 17:15 | ||
| Presentation | Poster | ||
| Topic | Image, Video, and Multidimensional Signal Processing: [IVARS] Image & Video Analysis, Synthesis, and Retrieval | ||
| IEEE Xplore Open Preview | Click here to view in IEEE Xplore | ||
| Abstract | In this paper, we propose a novel method to learn face sketch synthesis models by using unpaired data. Our main idea is bridging the photo domain $\mathcal{X}$ and the sketch domain $Y$ by using the line-drawing domain $\mathcal{Z}$. Specially, we map both photos and sketches to line-drawings by using a neural style transfer method, i.e. $F: \mathcal{X}/\mathcal{Y} \mapsto \mathcal{Z}$. Consequently, we obtain \textit{pseudo paired data} $(\mathcal{Z}, \mathcal{Y})$, and can learn the mapping $G:\mathcal{Z} \mapsto \mathcal{Y}$ in a supervised learning manner. In the inference stage, given a facial photo, we can first transfer it to a line-drawing and then to a sketch by $G \circ F$. Additionally, we propose a novel stroke loss for generating different types of strokes. Our method, termed sRender, accords well with human artists' rendering process. Experimental results demonstrate that sRender can generate multi-style sketches, and significantly outperforms existing unpaired image-to-image translation methods. | ||