Using chatbots to support online students' engagement and performance

Abstract

The prominence of online learning has increased over the past two decades with the advance of online technology. In such an online learning environment, all instruction and assessments are conducted virtually. For example, students watch recorded instructional videos and read references uploaded to the learning management system. Despite its growing popularity and advantages of providing convenience, online learning suffers persistently from low student engagement (e.g., students not participating in or dropping out of learning activities). One reason for students’ reluctance to complete learning tasks on their own is the lack of immediate guidance or help from teachers or peers when students are isolated and unable to understand the domain knowledge. This research aims to respond to this challenge by exploring the innovative use of chatbots to enhance students' online learning in a mixed-methods multiple case study, which involved two classes of students experiencing chatbot-supported online learning. Study 1 involved an 8-week undergraduate information management science course with 41 junior students, while Study 2 entailed a 12-week undergraduate English as a Foreign Language listening course with 46 freshmen. Specifically, we developed and integrated a chatbot named Learning Buddy as a learning partner for both cases to guide students in completing the online activities while communicating with them in real time. To facilitate meaningful interaction between the chatbots and students, we formulated the chatbot conversation design based on the Community of Inquiry framework (CoI) that includes social presence, cognitive presence, and teaching presence. We evaluated students’ learning engagement, performance, and perceptions of chatbot-supported online learning. Student engagement was examined from three aspects: behavioral engagement (e.g., participation in activities), cognitive engagement (e.g., quality of students’ learning artefacts), and emotional engagement (e.g., interest in learning). Learning performance was measured by pre- and post-tests regarding the domain knowledge. Students’ perceptions were measured by the CoI survey to assess their opinions toward each of the CoI presences existing in chatbot-supported online courses. We also conducted open-ended surveys and follow-up interviews with students to explore their perceptions of interacting with the chatbots and their suggestions for future educational chatbot design. In terms of student engagement, the results suggested that most students maintained conversation with the chatbots to complete the learning tasks with satisfactory quality, and they showed a relatively high level of interest in the chatbots throughout the learning process. Results also indicated that the chatbots facilitated students’ learning performance regarding the domain knowledge, as evidenced by the significantly increased scores in the posttests. Benefits that contributed to such positive results included the chatbots' instant interaction and personalized feedback. In addition, students perceived that the chatbot underpinned by the CoI framework could converse with them in a comfortable way while providing clear instructions on how to participate in the online activities and encouraging them to explore learning materials related to the online courses. Based on the research findings along with relevant theoretical and empirical support, we proposed a set of design principles for chatbot-supported online learning.