Joint Modeling of Big Five and HEXACO for Multimodal Apparent Personality-trait Recognition

2025-10-15

AI Summary

Prior work in computational personality analysis has largely overlooked critical traits such as Honesty-Humility, limiting theoretical and practical validity. Method: This study pioneers the joint modeling of the HEXACO model—including the Honesty-Humility dimension—with the classical Five-Factor Model (FFM) for multimodal apparent personality recognition. We propose a unified optimization framework that fuses linguistic, facial expression, and body pose features from self-introduction videos, explicitly encoding structural relationships between the five FFM and six HEXACO dimensions. Results: Experiments on a benchmark self-introduction video dataset demonstrate significant improvements in prediction accuracy across all dimensions for both models, with particularly substantial gains for Honesty-Humility. Moreover, our framework uncovers nonlinear mapping relationships between the HEXACO six-factor and FFM five-factor structures, establishing a novel, theory-grounded, and interpretable paradigm for cross-model personality representation.

Technology Category

• Machine Learning: Multimodal Learning
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior
• Application Domains: Humanities & Computational Social Science

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

This paper proposes a joint modeling method of the Big Five, which has long been studied, and HEXACO, which has recently attracted attention in psychology, for automatically recognizing apparent personality traits from multimodal human behavior. Most previous studies have used the Big Five for multimodal apparent personality-trait recognition. However, no study has focused on apparent HEXACO which can evaluate an Honesty-Humility trait related to displaced aggression and vengefulness, social-dominance orientation, etc. In addition, the relationships between the Big Five and HEXACO when modeled by machine learning have not been clarified. We expect awareness of multimodal human behavior to improve by considering these relationships. The key advance of our proposed method is to optimize jointly recognizing the Big Five and HEXACO. Experiments using a self-introduction video dataset demonstrate that the proposed method can effectively recognize the Big Five and HEXACO.

Problem

• Jointly modeling Big Five and HEXACO personality traits
• Recognizing apparent personality from multimodal human behavior
• Clarifying relationships between Big Five and HEXACO via machine learning

Innovation

• Joint modeling of Big Five and HEXACO traits
• Multimodal recognition from human behavior
• Optimized joint recognition through machine learning