Successful (and unsuccessful) features of academic group oral discussion: using learner corpora to explore language assessment

Abstract

In this paper, a corpus of 59, 20-25-minute long, 5-person group oral EAP assessments spanning 20 hours and 150,309 words of L1 Hong Kong L2 English learner was constructed. The data, graded by teacher-raters as between A to C grades using an in-house can-do scale for successful academic stance, interaction and comprehensibility, was annotated for 22 error types, an exhaustive range of interactive and interpersonal metadiscourse (following Hyland, 2005) and a range of temporal, prosodic, lexical and syntactic markers (or 'fluencemes') of (dis)fluency (following Götz, 2013). The results suggest that successful, frequent use of metadiscourse (particularly engagement markers and topic shifts) is the primary indicator of raters' positive evaluation of student performance in L2 academic tutorial discussion alongside temporal fluencemes (namely speech rate per minute). Identical repeats and errors of idiom / collocation are salient to raters' negative appraisals, while other L2 errors and other individual prosodic (e.g. pauses), lexical (e.g. reformulations) and syntactic fluencemes (e.g. interrupted structures, dependent clauses) do not particularly feature in raters' positive (or negative) evaluations. The detailed cross-sectional data afforded via this kind of corpus analysis serves as quantitative evidence of the linguistic features involved in grading decisions across the rubric, which can (and should) be used in discussions of standardisation and moderation for the raters involved.