Assessing collaborative problem solving (CPS) : task design contexts

Abstract

Today, students increasingly participate in teams to learn and solve problems together. Collaborative problem solving (CPS) competence has been recognized as an important 21st century educational outcome. In support of this, assessment of CPS skills has become critical. There are two key purposes underpinning CPS assessment: to support collaborative learning and to measure CPS as an attainment. The former looks for evidence of CPS outcomes in different learning design contexts to gain insights into how different task designs, interactions, and pedagogical settings, including group composition, technology, and other environmental support features influence students’ CPS performance. Open-ended, authentic task settings are generally preferred in educational settings for fostering CPS skills. The latter measures students’ CPS performance as an educational outcome and structured tasks are typically used for such purpose. To compare CPS achievement across students for summative purposes, the assessment needs to be able to measure CPS as a relatively stable attribute that is independent of task and other contextual settings. By assessing students’ CPS performance in open-ended, authentic task settings and structured, virtual task settings, this research aims to identify how task settings might affect CPS performance. This will provide a deeper understanding of the extent to which CPS performance as evidenced in learning situations and as measured in summative assessment contexts are consistent. Another strand of inquiry in the present study is to explore the potential use of serious games to support both learning and assessment of CPS. Serious games have the advantage of supporting more authentic CPS interactions that are structured and still maintain some level of openness. Three sets of case studies have been conducted to explore a) whether and how CPS performance varies across problem solving stages in an open-ended design task extended over time, and whether these assessed outcomes are consistent with the students’ performance as assessed by the ATC21S CPS assessment system; b) whether and how task settings influence CPS behaviours; and c) whether there are differences in how prior knowledge and experience influence students’ CPS performance between task settings. Each of the three studies engaged students in CPS in an open-ended, authentic task setting and a structured, virtual task setting. For the former, students collaborated to design/enhance a serious game. For the latter, students took the ATC21S CPS tests in the first study, and played a serious game in the other two studies. The results show differences in students’ CPS performance across task settings in all three studies, and that group dynamics, group composition, problem solving stage, and prior knowledge and experience matter. Further, CPS assessment should consider both individual and group performance. The findings also show the potential advantage of using serious games to deliver CPS assessment tasks that are flexible, more open-ended, and extended over time in structured, virtual task settings. This contributes to the theory and practice in the design of CPS assessment. In particular, it has shown the potential for serious games to be a viable technology environment to support both learning and assessment of CPS.