Exploring rater conceptions of academic stance and engagement during group tutorial discussion assessment

Abstract

The present study uses concurrent think-aloud verbal protocols alongside post-hoc interviews to explore how six teacher-raters determine a students' ability to explain academic concepts and argue for an academic stance supported by sources during a 25-minute group tertiary academic tutorial oral assessment. We explored how the raters arrived at decisions regarding the quality of students' academic stance and engagement in light of difficulties with rater attention in real-time, L2 language concerns, assessing engagement in a group oral setting, and the use of spoken citation to support speakers’ claims. Substantial differences in rater practice, beliefs and interpretation of assessment criteria were all found during the assessment of student performance, confirming a number of difficulties faced by raters assessing the academic ability of multiple participants over lengthy extended, interactional discourse. The findings shed real-time conceptions of (un)successful academic stance and engagement in group oral contexts, as well as confirm the usefulness of verbal protocols in revealing previously hidden complications for group oral assessments in an academic context, with accompanying suggestions for resolving such complications.