Face Swapping for Low-Resolution and Occluded Images In-the-Wild

Abstract

Safeguarding personal identity in various surveillance videos, dashcams, and on-street videos is crucial. One way to do this is to detect faces and blur them, but a better solution is to replace them with non-existent ones to maintain the naturalness of the videos. While face swapping methods have already been used in the media industry with high-quality faces, it is challenging to apply them for identity protection to faces in-the-wild where faces are often occluded and of low-resolution. Therefore, we propose a new framework for face swapping specifically designed to work with face images taken in real-world scenarios, making it useful as a privacy protection method. To tackle the issue of low-resolution images, we introduce a Cross-Resolution Contrastive Loss (CRCL) technique, which allows our neural network model to be trained using triplets of varying resolutions. This enables the model to learn and use identity information across different resolutions, thereby improving its accuracy. We also propose a plug-and-play framework that can be easily applied to existing face swapping models to handle occlusions. By explicit swapping of facial features and filling of occluded regions, our framework provides a more seamless blend. To demonstrate the effectiveness of our method in handling faces in-the-wild, we create an occluded VGGFace2 dataset consisting of face images augmented with various facial masks and hand occlusions. Through quantitative and qualitative assessments on this dataset, our proposed method demonstrates robust performance under low-resolution or occluded scenarios. Significant improvements are made in the quality of swapped faces while preserving their identity and attributes, highlighting the effectiveness of our framework in advancing face swapping as a reliable privacy protection measure.