Summative clinical Assessments: tensions between authenticity, reliability and assessment 'for' learning

Abstract

Aims - The role of summative clinical performance tasks in an assessment ‘for’ learning philosophy potentially broadens thinking about their use. The aim of this study was to qualitatively explore student and staff perceptions of standards, judgment practices and feedback in summative clinical assessments. Materials and Methods - Interviews were recorded with 3 focus groups of undergraduate dental students (n=24) and 6 individual clinical academic staff. IRB approval was obtained. Interviews were transcribed and a thematic analysis was performed. Results - Two key themes were identified: the tension between validity and reliability; and the role of assessment for learning. Positive and negative aspects were identified. Students and clinical staff recognized the value of the high-stakes summative clinical skills assessments as authentic and requisite for professional gatekeeping. While supporting the assessment for learning agenda, this perceived high validity, also raised complex issues regarding reliability. These issues included: task design, case complexity, assessor values and judgment practices, and inter-rater reliability. Student stress was seen as negative in terms of preparation and the procedure itself as well as issues of fairness in final judgments; however, stress was seen as positive with feedback on failure cases enhancing learning. Standards setting were perceived by staff as helpful in supporting reliability; however, counter to expectation, students did not value these as strongly nor did they perceive these to be part of an assessment for learning framework. Additionally, students felt feedback processes were variable with written comments perceived as mainly for faculty validation rather than for self-improvement. Conclusion – These highly valid, authentic assessments are required for determining competence and facilitating learning. Assessment managers should consider task design and clearly communicate aims and expectations to students, for example, through the use of exemplars and calibration practices. In addition, formative feedback should be detailed for such summative tasks.