Understanding eye movements in face processing using hidden Markov models

Abstract

The importance of eye movements in face processing has been demonstrated in recent studies, but many of them did not take individual differences in eye movements into account. To better understand how individual differences in eye movements modulate face processing, the studies reported in this thesis used hidden Markov models (HMMs) to analyze eye movements in face recognition and face preference tasks. The HMM approach is able discover individually different eye movement patterns in a data-driven fashion; it also can reflect both the spatial and the temporal aspects of eye movements, while many other methods are only able to reflect the spatial aspect. In the first study (Chapter 2), the advantage of the HMM approach was demonstrated by analyzing the eye movements recorded in a face recognition study. It was found that participants showed holistic and analytic patterns. The second study (Chapter 3) examined whether different eye movement patterns during face recognition were associated with different ethnicities and different performance. It was found that ethnicity did not play a significant role in modulating eye movement pattern, but the analytic pattern was associated with significantly better performance. In the third study (Chapter 4), the relationship between eye movements and performance was examined in further details; the eye movements during the learning phase and the recognition phase of the face recognition experiment were analyzed. It was found that the analytic pattern during the learning phase and the recognition phase were associated with significantly better performance, but whether participants switched pattern for the two phases did not modulate their performance. In the final study (Chapter 5), the relationship between eye movements and face preferences were examined through the analysis of the eye movement data collected in a two-alternative-forced-choice task. It was found that participants showed different strength of bias toward the preferred alternatives than the non-preferred alternatives. Participants’ preferences were successfully inferred based on their eye movements alone. Overall, these findings demonstrated the importance of considering individual differences in eye movements and the advantage of using the HMM approach to analyze eye movement data.