Preserving authenticity: transfer learning methods for detecting and verifying facial image manipulation

Kịnal R Sheth; Vishal S Vora

doi:10.15625/2525-2518/18626

Author affiliations

Authors

Kịnal R Sheth Assistant Professor, Electronics & Communication, L D College of Engineering, Ahmedabad - 380052 Gujarat, India https://orcid.org/0000-0003-0132-3533
Vishal S Vora Head & Associate Professor, Atmiya Institute of Technology, Atmiya University, Gujarat, India https://orcid.org/0000-0002-5111-6682

DOI:

https://doi.org/10.15625/2525-2518/18626

Keywords:

Image Retouching, deep learning, image classification, Machine Learning, transfer learning

Abstract

Facial retouching in supporting documents can have adverse effects, undermining the credibility and authenticity of the information presented. This paper presents a comprehensive investigation into the classification of retouched face images using a fine-tuned pre-trained VGG16 model. We explore the impact of different train-test split strategies on the performance of the model and also evaluate the effectiveness of two distinct optimizers. The proposed fine-tuned VGG16 model with “ImageNet” weight achieves a training accuracy of 99.34 % and a validation accuracy of 97.91 % over 30 epochs on the ND-IIITD retouched faces dataset. The VGG16_Adam model gives a maximum classification accuracy of 96.34 % for retouched faces and an overall accuracy of 98.08 %. The experimental results show that the 50 % - 25 % train-test split ratio outperforms other split ratios mentioned in the paper. The demonstrated work shows that using a Transfer Learning approach reduces computational complexity and training time, with a max. training duration of 39.34 min for the proposed model.

Downloads

Download data is not yet available.

References

Russello S. - The Impact of Media Exposure on Self-Esteem and Body Satisfaction in Men and Women, Vol. 1, 2009.

Gupta S. - JIPR 10 (6) (2005) 491-498.

Altabe M. - Ethnicity and body image: Quantitative and qualitative analysis, Int. J. Eat. Disord. 23 (2) (1998) 153-159. doi: 10.1002/(SICI)1098-108X(199803)23:2<153::AID-EAT5>3.0.CO;2-J.

Kee E. and Farid H. - A perceptual metric for photo retouching, Proc. Natl. Acad. Sci. U. S. A. 108 (50) (2011) 19907-19912. doi: 10.1073/pnas.1110747108.

Kose N., Apvrille L., and Dugelay J. L. - Facial makeup detection technique based on texture and shape analysis, 2015 11th IEEE Int. Conf. Work. Autom. Face Gesture Recognition, FG 2015, 2015, doi: 10.1109/FG.2015.7163104.

Bharati A., Singh R., Vatsa M., and Bowyer K. W. - Detecting Facial Retouching Using Supervised Deep Learning, IEEE Trans. Inf. Forensics Secur. 11 (9) (2016) 1903-1913. doi: 10.1109/TIFS.2016.2561898.

Bharati A., Vatsa M., Singh R., Bowyer K. W., and Tong X. - Demography-based facial retouching detection using subclass supervised sparse autoencoder, arXiv, 2017.

Singh A., Tiwari S., and Singh S. K. - Face tampering detection from single face image using gradient method, Int. J. Secur. its Appl. 7 (1) (2013) 17-30.

Rathgeb C., et al. - PRNU-based detection of facial retouching ISSN 2047-4938, IET Biometrics 9 (4) (2020) 154-164. doi: 10.1049/iet-bmt.2019.0196.

Seibold C., Hilsmann A., and Eisert P. - Reflection Analysis for Face Morphing Attack Detection, 2018 26th Eur. Signal Process. Conf., 2018, pp. 1022-1026.

Ciftci U. A. - FakeCatcher : Detection of Synthetic Portrait Videos using Biological Signals, Vol. X, no. X, 2020, pp. 1-17.

Akhtar Z., Dasgupta D., and Banerjee B. - Face Authenticity : An Overview of Face Manipulation Generation , Detection Available on : Elsevier-SSRN Face Authenticity : An Overview of Face Manipulation Generation , Detection and Recognition, May, 2019.

Sharma K., Singh G., and Goyal P. - IPDCN2 : Improvised Patch-based Deep CNN for facial retouching detection, Vol. 211, May 2021, 2023.

Alzubaidi L., et al. - Review of deep learning : concepts , CNN architectures , challenges, applications , future directions, Springer International Publishing, 2021.

Krishna S. T. and Kalluri H. K. - Deep learning and transfer learning approaches for image classification, Int. J. Recent Technol. Eng. 7 (5) (2019) 427-432.

Desai C. G. and N. Academy D. - Image Classification Using Transfer Learning and Deep Learning, September 2021. doi: 10.18535/ijecs/v10i9.4622.

Ying Q., Liu J., Li S., Xu H., Qian Z., and Zhang X. - RetouchingFFHQ : A Large-scale Dataset for Fine-grained Face Retouching Detection.”

Bichri H., Chergui A., and Hain M. - ScienceDirect Image Classification with Transfer Learning Using a Custom Dataset : Comparative Study Image Classification with Transfer Learning Using a Custom Dataset : Comparative Study, Procedia Comput. Sci. 220 (2023) 48-54. doi: 10.1016/j.procs.2023.03.009.

Ibrahim A. M., Elbasheir M., Badawi S., Mohammed A., and Alalmin A. F. M. - Skin Cancer Classification Using Transfer Learning by VGG16 Architecture (Case Study on Kaggle Dataset), 2023, pp. 67-75. doi: 10.4236/jilsa.2023.153005.

Sheth K. R. and Vora V. S. - A comparative study on image forgery-facial retouching 12 (2) (2023) 851-859. doi: 10.11591/eei.v12i2.4481.

Shyu M., Chen S., and Iyengar S. S. - A Survey on Deep Learning : Algorithms, Techniques, ACM Comput. Surv. 51 (5) (2018) 1-36.

Sharma N. and Sharma N. - An Neural An Analysis Analysis Of Of Convolutional Convolutional Neural Networks Networks For For Image Image An Analysis of Co Classification an Analysis of Convolutional Neural Networks for Image Classification an Analysis of Convolutional Neural and Ne, Procedia Comput. Sci. 132 (Iccids) (2018) 377-384. doi: 10.1016/j.procs.2018.05.198.

Simonyan K. and Zisserman A. - Very deep convolutional networks for large-scale image recognition, 3rd Int. Conf. Learn. Represent. ICLR 2015 - Conf. Track Proc., 2015, pp. 1-14.

face Database N. I. R. - https://cvrl.nd.edu/projects/data/. Please replace by other suitable ref., for example, journal, book, proceeding, etc.

Kandel I. and Castelli M. - Comparative Study of First Order Optimizers for Image Classification Using Convolutional Neural Networks on Histopathology Images, 2020.

Jain A., Singh R., and Vatsa M. - On detecting GANs and retouching based synthetic alterations, 2018. doi: 10.1109/BTAS.2018.8698545.