文献类型：会议论文

英文题名：Swin-CasUNet: Cascaded U-Net with Swin Transformer for Masked Face Restoration

作者：Zeng, Chengbin Liu, Yi Song, Chunli

第一作者：曾成斌

通信作者：Zeng, CB[1]

机构：[1]Guizhou Inst Technol, Sch Big Data, Guiyang 550003, Guizhou, Peoples R China

第一机构：贵州理工学院

通信机构：corresponding author), Guizhou Inst Technol, Sch Big Data, Guiyang 550003, Guizhou, Peoples R China.|贵州理工学院;

会议论文集：26th International Conference on Pattern Recognition / 8th International Workshop on Image Mining - Theory and Applications (IMTA)

会议日期：AUG 21-25, 2022

会议地点：Montreal, CANADA

语种：英文

外文关键词：Computer vision - Image reconstruction - Network architecture

年份：2022

摘要：Masked face restoration is one of the most valuable challenges in the computer vision community. With the in-depth study of u-shaped architectures, also known as U-Net, great progress has been achieved in the development of masked face restoration during the past few years. However, previous restoration methods fail to fully model the long-range dependency due to the locality of convolution layers of the U-Net. To address this problem, we propose a shifted windows Transformer (Swin Transformer) based cascaded U-Net framework called Swin-CasUNet, which incorporates the long-range dependency merit of Transformer into the cascaded U-Net architecture to effectively enhance the functionality and generalization of Ushaped architecture. Specifically, we design a two-stage cascaded U-Net architecture to implement the coarse-to-fine restoration of the masked face. Swin Transformers is adopted to extract global self-attention contexts for the feature map produced by the encoder part of the U-Net. An improved face structure loss is proposed to supervise structure learning. To evaluate the robustness of our masked face restoration model, we collect 3800 pairs of full face images and corresponding masked face images from the real-world and web. Experiments on the datasets demonstrate that our proposed method can generate high quality restoration results. In order to quantitatively compare with previous face restoration methods, we modify the input of our system by manually adding regular and irregular white masks on CelebA face datasets, and then retrain our network. Experiments show that our Swin-CasUNet outperforms previous methods on benchmark datasets.