Discourse and perceptual analysis of AI-synthesized texts on coherence and cohesion

Abstract

 Technology has been an integrated component in language learning in the modern age. Immense changes have been brought forward to the learning strategies in face of technological advancements. Breakthroughs in the development of Artificial Intelligence (AI) in recent years have attracted contemplations on new opportunities or threats to our human society. With the latest innovations in Natural Language Process models, the AI has developed the competence of content creation, including the generation of texts in written paragraphs. This emergence of a new source of language production may have an impact on the human language learning process. Against this background, this study investigates the development of coherence and cohesion in AI-generated texts created by the GPT-2 NLP model. Ninety AI-generated written English samples in three text types, namely formal email, debate speech and short story, were analysed through the perspectives of the textual framework of Systematic Functional Linguistics (SFL) and human perception. Textual analysis was carried out using thematic progression analysis and the evaluation of cohesive strategies identified in the lexical, grammatical and logical aspects. Meanwhile, interviews were conducted with nine adult informants from different fields of education and varied experience teaching or learning the English language for the analysis of perceived coherence and cohesion. The analysis identified signs of coherence with the existence of four types of theme-rheme connections, as well as cohesive devices in the three aspects, manifested to different extents in the three text types. The perceptual data revealed mixed responses to the AI-generated texts, including flaws in the development of a logical flow of ideas and the mechanical use of lexico-grammatical resources, leading to sub-standard coherence and cohesion. However, observations were made on instances of human-like written features despite imperfection in the language use. While the results suggest the potentials of AI to generate texts with a certain level of readability, future studies may expand the research scope with other SFL metafunctions, a broader range of texts synthesized by various AI models, and investigation of human perception with a different population.