Using Multimodal Learning Analytics to Examine Learners' Responses to Different Types of Background Music during Reading Comprehension

Abstract

Previous studies have evaluated the affordances and challenges of performing cognitively demanding learning tasks with background music (BGM), yet the effects of various types of BGM on learning still remain an open question. This study aimed to examine the impacts of different music genres and fine-grained music characteristics on learners' emotional, physiological, and pupillary responses during reading comprehension. Leveraging multimodal learning analytics (MmLA) methods of collecting data in multiple modalities from learners, a user experiment was conducted on 102 participants, with half of them reading with self-selected BGM (i.e., the experimental group), while the other half reading without BGM (i.e., the control group). Results of statistical analyses and interviews revealed significant differences between the two groups in their self-reported emotions and automatically measured physiological responses when the experimental group was exposed to classical, easy-listening, rebellious and rhythmic music. Fine-grained music characteristics (e.g., instrumentation, tempo) could predict learners' emotions, pupillary, and physiological responses during reading comprehension. The expected contributions of this study include: 1) providing empirical evidence for understanding affective dimensions of learning with BGM, 2) applying MmLA methods for examining the impacts of BGM on learning, and 3) yielding practical implications on how to improve learning with BGM.