The differences in essential facial areas for impressions between humans and deep learning models
An eye‐tracking and explainable AI approach
Bibliographic Data
| ID | 9619491 |
|---|---|
| Authors | Toshiyuki Sano (0000-0002-8811-1420, Graduate School of Human Relations Keio University Minato‐ku Tokyo Japan, corresponding author), Jun Shi (0000-0001-7718-7983, Graduate School of Human Relations Keio University Minato‐ku Tokyo Japan), Hideaki Kawabata (0000-0001-8132-4995, Graduate School of Human Relations Keio University Minato‐ku Tokyo Japan) |
| Year | 2024 |
| Volume | 117 |
| Issue | 2 |
| Pages | 503-527 |
| Publication date | 2024-10-25 |
| Open Access | Yes |
| Type | ARTICLE |
| Venue | British Journal of Psychology (JOURNAL) |
| Journal identifiers | ISSN: 0007-1269 • E-ISSN: 2044-8295 |
| Publisher | Wiley (PUBLISHER • GB) |
| DOI | 10.1111/bjop.12744 |
| PMID | 39460393 |
| OpenAlex | W4403793220 |
| Language | EN |
| Citations received | 3 |
| References cited | 71 |
This study explored the facial impressions of attractiveness, dominance and sexual dimorphism using experimental and computational methods. In Study 1, we generated face images with manipulated morphological features using geometric morphometrics. In Study 2, we conducted eye tracking and impression evaluation experiments using these images to examine how facial features influence impression evaluations and explored differences based on the sex of the face images and participants. In Study 3, we employed deep learning methods, specifically using gradient‐weighted class activation mapping (Grad‐CAM), an explainable artificial intelligence (AI) technique, to extract important features for each impression using the face images and impression evaluation results from Studies 1 and 2. The findings revealed that eye‐tracking and deep learning use different features as cues. In the eye‐tracking experiments, attention was focused on features such as the eyes, nose and mouth, whereas the deep learning analysis highlighted broader features, including eyebrows and superciliary arches. The computational approach using explainable AI suggests that the determinants of facial impressions can be extracted independently of visual attention
Cognitive psychology · Cognitive science · Eye tracking · Tracking (education) · Artificial Intelligence · Computer Science · Evolutionary Psychology and Human Behavior · Face recognition and analysis · Face Recognition and Perception · Psychology
Facial attractiveness
Attractive Faces Are Only Average
Social inferences from faces
Variability in photos of the same face
Effects of sexual dimorphism on facial attractiveness
Shape Manifolds, Procrustean Metrics, and Complex Projective Spaces
The Chicago face database
Facial attractiveness
The intrinsic memorability of face photographs.
A saliency-based search mechanism for overt and covert shifts of visual attention
Human (Homo sapiens) facial attractiveness and sexual selection
Principal warps
The functional basis of face evaluation
Simr
The effect of aging on facial attractiveness
Which Facial Features Are Central in Impression Formation
Facial Image Manipulation
Geometric morphometrics of male facial shape in relation to physical strength and perceived attractiveness, dominance, and masculinity
Women’s Mating Strategies and Mate Value Are Associated with Viewing Time to Facial Masculinity
The Evolutionary Psychology of Facial Beauty
Face gender and stereotypicality influence facial trait evaluation
| Citations per year | 3 |
|---|---|
| Citation span | 2026 - 2026 (1) |
| Highly cited | No |