Forensic investigation and detection on deepfake using deep learning methods

Abstract

The fast development of Deepfake has drawn considerable public attention due to security and privacy concerns in social media digital forensics. Due to its free-accessed source code and applications, Deepfake has brought huge current and potential future negative impacts to our daily lives, and even anyone can become a potential victim. As the circulating popular Deepfake videos become more realistic, it is difficult to distinguish between real and fake by human eyes. Therefore, a reliable and efficient automatic Deepfake detection approach is urgently desired.

While the traditional detection techniques have failed in distinguishing between real and fake, various Deepfake detection approaches have been attempted using deep learning models. However, the performance of the existing approaches is not satisfied to be applied in real-life Deepfake challenges. First of all, the black-box characteristic makes most existing methods unexplainable and the actual underlying Deepfake forensic traces have been barely discussed. Secondly, the existing well-trained deep learning models mostly perform poorly on unseen Deepfake datasets with unknown facial manipulation techniques regarding the training dataset. Lastly, despite being evaluated on various benchmark datasets, the current experimental setting does not correspond to the real-life Deepfake distribution, thus the models are mostly unreliable to be utilized and applied as evidence at the court.

This thesis conducts three studies to resolve the three stated problems. In the first study, we investigate the special forensic noise traces within Deepfake images and propose a noise-based Deepfake detection model approach using a deep neural network. The visualization of the Deepfake forensic noise traces shows explicit distinctions between synthesized faces and any unmodified area. In the second study, we propose a deep convolutional Transformer model to incorporate the decisive image features both locally and globally. In specific, we apply convolutional pooling and re-attention to enrich the extracted general image features and enhance the model transferability on unseen datasets. In the last study, we propose a lightweight deep learning Deepfake detection approach and conduct a model reliability study using statistical random sampling from the available benchmark datasets to imitate the real-life Deepfake cases. A sufficient number of trials for model evaluation with random sampling derives the 95% and 90% confidence intervals, informing the reliable accuracy information of the proposed model.