Deepfake image detection using a deep hybrid convolutional neural network

Bagherzadeh, Fahimeh; Rastgoo, Razieh

doi:10.22075/jme.2023.31438.2511

Deepfake image detection using a deep hybrid convolutional neural network

Document Type : Computer Article

Authors

¹ B.Sc. Student, Faculty of Electrical and Computer Engineering, Semnan University, Semnan, Iran

² Assistant Professor, Faculty of Electrical and Computer Engineering, Semnan University, Semnan, Iran

10.22075/jme.2023.31438.2511

Abstract

Deepfake refers to a category of fake and artificial data in which fake content is produced based on existing content. This content can include image, video and audio signals. Deepfake production is based on deep generative networks that manipulate data or produce fake images and videos. In recent years, many studies have been conducted to understand how deepfakes work, and many methods based on deep learning have been introduced to identify videos or images produced by deepfakes and distinguish them from real images. In order to improve the accuracy of deep-fake detection and simultaneously use the capabilities of different types of convolutional neural networks, in this article, a hybrid model is presented using four convolutional neural networks: DenseNet201, EfficientNetB2, Inception-ResNet-V2, and ResNet152. turns Relying on the high capabilities of these networks in extracting effective features from the input image, the proposed model is able to simultaneously recognize whether the input image is deep or not by these four models. The results presented on the three databases of 140k real and fake faces, DFDC faces and Deepfake and real images indicate the improvement of the results compared to the existing models.

Keywords

Main Subjects

Computer Engneering

References

[1] Zobaed, Sm, and Md Fazle Rabby, Md Istiaq Hossain, Ekram Hossain, Sazib Hasan, Asif Karim, and Khan Md. Hasib. "Deepfakes: Detecting forged and synthetic media content using machine learning". Artificial Intelligence in Cyber Security: Impact and Implications (2021): 177–201.

[2] Rastgoo, Razieh and Vahid Sattari Naeini. "A neurofuzzy QoS-aware routing protocol for smart grids". 22nd Iranian Conference on Electrical Engineering (ICEE), pp. 1080-1084, 2014.

[3] Rastgoo, Razieh and Vahid Sattari Naeini. "Tuning parameters of the QoS-aware routing protocol for smart grids using genetic algorithm". Applied Artificial Intelligence 30, no. 1 (2016): 52-76.

[4] Majidi, Neza, Kourosh Kiani, and Razieh Rastgoo. "A deep model for super-resolution enhancement from a single image". Journal of AI and Data Mining 8, no. 4, (2020): 451-460.

[5] Kiani, Kourosh, Razieh Hematpour, and Razieh Rastgoo. "Automatic grayscale image colorization using a deep hybrid model". Journal of AI and Data Mining 9, no. 3 (2021): 321-328.

[6] Rastgoo, Razieh and Vahid Sattari-Naeini. "Gsomcr: Multi-constraint genetic-optimized qos-aware routing protocol for smart grids". Iranian Journal of Science and Technology, Transactions of Electrical Engineering 42, (2018): 185-194.

[7] Rastgoo, Razieh and Kourosh Kiani. "Face recognition using fine-tuning of Deep Convolutional Neural Network and transfer learning". Journal of Modeling in Engineering 17, no. 58 (2019): 103-111.

[8] Rastgoo, Razieh, Kourosh Kiani, Sergio Escalera, and Mohammad Sabokrou. "Multi-modal zero-shot sign language recognition". arXiv:2109.00796, (2021).

[9] Zarbafi, Sahar, Kourosh Kiani, and Razieh Rastgoo. "Spoken Persian digits recognition using deep learning". Journal of Modeling in Engineering 21, (2023): 163-172.

[10] Alinezhad, Fatemeh, Kourosh Kiani, and Razieh Rastgoo. "A Deep Learning-based Model for Gender Recognition in Mobile Devices". Journal of AI and Data Mining 11, (2023): 229-236.

[11] Rastgoo, Razieh, Kourosh Kiani, and Sergio Escalera. "ZS-SLR: Zero-Shot Sign Language Recognition from RGB-D Videos". arXiv:2108.10059, (2021).

[12] Thambawita, Vajira, and et al. "DeepFake electrocardiograms using generative adversarial networks are the beginning of the end for privacy issues in medicine". Sci. Rep. 11, (2021): 21869.

[13] Faisal Bin Ahmed, Mohammad, M. Saef Ullah Miah, Abhijit Bhowmik, and Juniada Binti Sulaiman. "Awareness to Deepfake: A resistance mechanism to Deepfake". In Proceedings of the 2021 International Congress of Advanced Technology and Engineering (ICOTEN), Taiz, Yemen, pp. 1–5, 2021.

[14] Gautam, Neil, and Dinesh Kumar Vishwakarma. "Obscenity Detection in Videos through a Sequential ConvNet Pipeline Classifie". IEEE Trans. Cogn. Dev. Syst. 15, no. 1 (2023): 310-318.

[15] Naik, Rakesh. "Deepfake Crimes: How Real and Dangerous They Are in 2021? ". Available online: https://cooltechzone.com/research/deepfake-crimes. Accessed Date: Jan 2024.

[16] Jin, Bo, Leandro Cruz, and Nuno Gonçalves. "Deep facial diagnosis: Deep transfer learning from face recognition to facial diagnosis". IEEE Access 8, (2020): 123649–123661.

[17] Ismail, Aya, Marwa Elpeltagy, Mervat S. Zaki, and Kamal Eldahshan. "A New Deep Learning-Based Methodology for Video Deepfake Detection Using XGBoost". Sensors 21, (2021): 5413.

[18] Chen, Weijun, Hongbo Huang, Shuai Peng, Changsheng Zhou, and Cuiping Zhang. "YOLO-face: A real-time face detector". Vis. Comput. 37, (2021): 805–813.

[19] Pan, Deng, Lixian Sun, Rui Wang, Xingjian Zhang, and Richard O. Sinnott. "Deepfake Detection through Deep Learning". In Proceedings of the 2020 IEEE/ACM International Conference on Big Data Computing, Applications and Technologies (BDCAT), Leicester, UK, pp. 134–143, 2020.

[20] Jung, Tackhyun, Sangwon Kim, and Keecheon Kim. "DeepVision: Deepfakes Detection Using Human Eye Blinking Pattern". IEEE Access 8, (2020): 83144-83154.

[21] Lewis, John K., Imad Eddine Toubal, Helen Chen, Vishal Sandesera, Michael Lomnitz, Zigfried Hampel-Arias, Calyam Prasad, and Kannappan Palaniappan. "Deepfake Video Detection Based on Spatial, Spectral, and Temporal Inconsistencies Using Multimodal Deep Learning". In Proceedings of the 2020 IEEE Applied Imagery Pattern Recognition Workshop (AIPR), Washington DC, USA, pp. 1–9, 2020.

[22] Hsu, Chih-Chung, Yi-Xiu Zhuang, and Chia-Yen Lee. "Deep fake image detection based on pairwise learning". Appl. Sci. 10, (2020): 370.

[23] Rana, Md. Shohel, and Andrew H. Sung. "DeepfakeStack: A Deep Ensemble-based Learning Technique for Deepfake Detection". In Proceedings of the 2020 7th IEEE International Conference on Cyber Security and Cloud Computing (CSCloud)/2020 6th IEEE International Conference on Edge Computing and Scalable Cloud (EdgeCom), New York, NY, USA, pp. 70–75, 2020.

[24] Huang, Gao, Zhuang Liu, Laurens van der Maaten, and Kilian Q. Weinberger. "Densely Connected Convolutional Networks". In Proceedings of the 2017 Computer Vision and Pattern Recognition (CVPR), Honolulu, Hawaii, 2017.

[25] Tan, Mingxing, and Quoc V. Le. "EfficientNet: Rethinking Model Scaling for Convolutional Neural Networks". In Proceedings of the 2019 International Conference on Machine Learning (ICML), Jun 2019.

[26] Szegedy, Christian, Sergey Ioffe, Vincent Vanhoucke, and Alex Alemi. "Inception-v4, Inception-ResNet and the Impact of Residual Connections on Learning". In Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence (AAAI'17), pp. 4278–4284, 2017.

[27] He, Kaiming, Xiangyu Zhang, Shaoqing Ren, and Jian Sun. "Deep Residual Learning for Image Recognition". In Proceedings of the 2017 Computer Vision and Pattern Recognition (CVPR), Caesars Palace, 2016.

[28] https://www.kaggle.com/datasets/xhlulu/140k-real-and-fake-faces. Access Date: Jan. 2024.

[29] https://www.kaggle.com/datasets/itamargr/dfdc-faces-of-the-train-sample, Access Date: Jan. 2024.

[30] https://www.kaggle.com/datasets/manjilkarki/deepfake-and-real-imagesv, Access Date: Jan. 2024.

[31] Raza, Ali, Kashif Munir, and Mubarak Almutairi, "A Novel Deep Learning Approach for Deepfake Image Detection". Appl. Sci. 12,( 2022): 9820.

[32] https://www.kaggle.com/code/abdalrhmanmorsi/real-vs-fake-face-cnn-model, Access Date: Jan. 2024.

[33] https://www.kaggle.com/code/shubhanshu609/deepfake-cnn-2, Access Date: Jan. 2024.