Towards Automated Analysis of Undergraduate Academic Writing using Metadiscourse, Cognitive Level and Word Network

Abstract

Automated assessment of students' academic writing can provide timely feedback and alleviate teachers' workload. This study examined 199 undergraduate students' essays through metrics of metadiscourse, cognitive levels, and word network, and the relationships between these metrics and performances. Metrics were calculated under the framework of Hyland's metadiscourse model and Bloom Taxonomy whereas relationships were revealed by correlation analysis and classification with multiple algorithms. Findings show that students employed more interactive metadiscourse markers than interactional ones, and there were weak to mild positive correlations between some of these metrics and performance score. High-performing essays involved more higher-order thinking and stronger connections among words. Besides, classification models combining all three types of metrics were most effective in differentiating high and low-performing essays. The three types of metrics evaluated can be potentially used to provide fine-grained suggestions for improving students' academic writing.