Analyzing and evaluating personality and human behavior based on facial index and big five model
Author affiliations
DOI:
https://doi.org/10.15625/1813-9663/18677Keywords:
Big Five, personality traits, CNN-LSTM, ResNet, VGG19, UDIVA dataset, facial landmarks.Abstract
Numerous studies have shown that morphological and social indicators in a human face can provide information about a person's personality and behavior. The Big Five model, also known as the Five-Factor, is the five basic dimensions of personality. These dimensions include openness, conscientiousness, extraversion, agreeableness and neuroticism. The Big Five model has been applied in a variety of different settings, including clinical psychology, organizational psychology, and even marketing research. By examining where an individual falls on each of these dimensions, researchers can gain insight into their unique personality traits and use this information to make predictions about their behavior and performance in different situations. In our existing iscv platform, a job searching website, we can help employers better understanding employee incentives by utilizing the personality traits information of candidates. Managers and CEOs can therefore discover a means to improve relationships and communication while also managing and building teams more effectively. We trained a machine learning model using a hybrid CNN-LSTM, ResNet, VGG19 algorithm for personality recognition through interview video. In each video, we analyze facial movement by using the 3D landmarks extracted with the 3DDFA-V2 algorithm. The model uses the UDIVA v0.5 dataset, collected in the scope of the research project entitled “Understanding Face-to-Face Dyadic Interactions through Social Signal Processing”. The experimental results conclude: (i) Analyzing facial movement by using the 3D landmarks extracted with the 3DDFA-V2 algorithm. (ii) Personality traits inferred from facial behaviors by most benchmarked deep learning model. (iii) Personality assessment model is trained from a combination of two data sets (one UDIVIA dataset and one self-survey dataset) to fit Asian personalities. (iv) The detailed Big Five personality tendency assessment table is based on the interview video and questionnaire of the surveyed people.
Metrics
References
M. Komarraju and S. Karau, “The relationship between the Big Five personality traits and academic motivation,” Personal. Individ. Differ., vol. 39, pp. 557–567, Aug. 2005, doi: 10.1016/j.paid.2005.02.013.
“The Big Five Personality Traits in the Political Arena | Annual Review of Political Science.” https://www.annualreviews.org/doi/abs/10.1146/annurev-polisci-051010-111659 (accessed Jun. 12, 2023).
L. Cai and X. Liu, “Identifying Big Five personality traits based on facial behavior analysis,” Front. Public Health, vol. 10, 2022, Accessed: Jun. 11, 2023. [Online]. Available: https://www.frontiersin.org/articles/10.3389/fpubh.2022.1001828
H.-Y. Suen, K.-E. Hung, and C.-L. Lin, “TensorFlow-Based Automatic Personality Recognition Used in Asynchronous Video Interviews,” IEEE Access, vol. 7, pp. 61018–61023, 2019, doi: 10.1109/ACCESS.2019.2902863.
M. Xue, X. Duan, Y. Wang, and Y. Liu, “A Computational Personality Traits Analysis Based on Facial Geometric Features,” in 2019 IEEE 14th International Conference on Intelligent Systems and Knowledge Engineering (ISKE), Nov. 2019, pp. 1107–1111. doi: 10.1109/ISKE47853.2019.9170334.
J. Xu, W. Tian, Y. Fan, Y. Lin, and C. Zhang, “Personality Trait Prediction Based on 2.5D Face Feature Model,” in Cloud Computing and Security, X. Sun, Z. Pan, and E. Bertino, Eds., in Lecture Notes in Computer Science. Cham: Springer International Publishing, 2018, pp. 611–623. doi: 10.1007/978-3-030-00021-9_54.
“Big Five Inventory.” https://psycnet.apa.org/doiLanding?doi=10.1037%2Ft07550-000 (accessed Jun. 12, 2023).
R. Bansal, G. Raj, and T. Choudhury, “Blur image detection using Laplacian operator and Open-CV,” in 2016 International Conference System Modeling & Advancement in Research Trends (SMART), Nov. 2016, pp. 63–67. doi: 10.1109/SYSMART.2016.7894491.
“FaceBoxes: A CPU real-time face detector with high accuracy | IEEE Conference Publication | IEEE Xplore.” https://ieeexplore.ieee.org/abstract/document/8272675 (accessed Jun. 12, 2023).
J. Guo, X. Zhu, Y. Yang, F. Yang, Z. Lei, and S. Z. Li, “Towards Fast, Accurate and Stable 3D Dense Face Alignment.” arXiv, Feb. 07, 2021. doi: 10.48550/arXiv.2009.09960.
E. W. Weisstein, “Rotation Matrix.” https://mathworld.wolfram.com/ (accessed Jun. 12, 2023).
C. Palmero et al., “Context-Aware Personality Inference in Dyadic Scenarios: Introducing the UDIVA Dataset.” arXiv, Dec. 28, 2020. doi: 10.48550/arXiv.2012.14259.
Open-Source Psychometrics Project, “The Big Five Personality Test”, August 2, 2019.
F. Pedregosa et al., “Scikit-learn: Machine Learning in Python,” J. Mach. Learn. Res., vol. 12, no. 85, pp. 2825–2830, 2011.
Komer, Brent, James Bergstra, and Chris Eliasmith. "Hyperopt-sklearn." Automated Machine Learning: Methods, Systems, Challenges (2019): 97-111.
Z. Wang and A. C. Bovik, “Mean squared error: Love it or leave it? A new look at Signal Fidelity Measures,” IEEE Signal Process. Mag., vol. 26, no. 1, pp. 98–117, Jan. 2009, doi: 10.1109/MSP.2008.930649.
M. R. Segal, “Machine Learning Benchmarks and Random Forest Regression,” Apr. 2004, Accessed: Jun. 12, 2023. [Online]. Available: https://escholarship.org/uc/item/35x3v9t4
F. Zhang and L. J. O’Donnell, “Chapter 7 - Support vector regression,” in Machine Learning, A. Mechelli and S. Vieira, Eds., Academic Press, 2020, pp. 123–140. doi: 10.1016/B978-0-12-815739-8.00007-9.
X. Su, X. Yan, and C.-L. Tsai, “Linear regression,” WIREs Comput. Stat., vol. 4, no. 3, pp. 275–294, 2012, doi: 10.1002/wics.1198.
A. V. Dorogush, V. Ershov, and A. Gulin, “CatBoost: gradient boosting with categorical features support.” arXiv, Oct. 24, 2018. doi: 10.48550/arXiv.1810.11363.
S. Hochreiter and J. Schmidhuber, “Long Short-term Memory,” Neural Comput., vol. 9, pp. 1735–80, Dec. 1997, doi: 10.1162/neco.1997.9.8.1735.
Downloads
Published
How to Cite
Issue
Section
License
1. We hereby assign copyright of our article (the Work) in all forms of media, whether now known or hereafter developed, to the Journal of Computer Science and Cybernetics. We understand that the Journal of Computer Science and Cybernetics will act on my/our behalf to publish, reproduce, distribute and transmit the Work.2. This assignment of copyright to the Journal of Computer Science and Cybernetics is done so on the understanding that permission from the Journal of Computer Science and Cybernetics is not required for me/us to reproduce, republish or distribute copies of the Work in whole or in part. We will ensure that all such copies carry a notice of copyright ownership and reference to the original journal publication.
3. We warrant that the Work is our results and has not been published before in its current or a substantially similar form and is not under consideration for another publication, does not contain any unlawful statements and does not infringe any existing copyright.
4. We also warrant that We have obtained the necessary permission from the copyright holder/s to reproduce in the article any materials including tables, diagrams or photographs not owned by me/us.
