Difficulties with computational coreference tracking: How to achieve ‘coherence in mind’ without a mind?

Abstract

The introduction and maintenance of reference in discourse is subject to a number of language-universal constraints on NP form and selection that allow speakers to maintain reference co-referentially (i.e. from mention to mention) with minimal processing effort across large spans of discourse. Automated approaches to co-reference resolution (i.e. the successful tracking of discourse referents across texts) use a variety of models to account for how a referent may be tracked co-referentially, yet automated co-reference resolution is still considered an incredibly difficult task for those working in natural language processing and generation, even when using ‘gold-standard’ manually-annotated discourse texts as a source of comparison. Less research has been conducted on the performance of automated co-reference resolution on narrative data, which is rich with multiple referents interacting with each other, with long sequences of continuous and non-continuous reference to be maintained. Five freely-available co-reference resolvers were trialled for accuracy on oral narrative data produced by English native speakers using a picture sequence as an elicitation device. However, accuracy levels of under 50% for each resolver tested suggest large differences between human and computational methods of co-reference resolution. In particular, zero anaphora, NPs with modifiers and errors in coding first-mentioned referents appear particularly problematic. This suggests that automated approaches to coreference resolution need a vast range of lexical knowledge, inferential capabilities based on situational and world knowledge, and the ability to track reference over extended discourse if they are to succeed in modelling human-like coreference resolution