Detecting suicide risk using knowledge-aware natural language processing and counseling service data

Abstract

Rationale: Detecting users at risk of suicide in text-based counseling services is essential to ensure that at-risk individuals are flagged and prioritized.

Objective: The objective of this study is to develop a domain knowledge-aware risk assessment (KARA) model to improve our ability of suicide detection in online counseling systems.

Methods: We obtained the largest known de-identified dataset from an emotional support system established in Hong Kong, comprising 5682 Cantonese conversations between help-seekers and counselors. Of those, 682 conversations disclosed crisis intentions of suicide. We constructed a suicide-knowledge graph, representing suicide-related domain knowledge as a computer-processible graph. Such knowledge graph was embedded into a deep learning model to improve its ability to identify help-seekers in crisis. As the baseline, a standard NLP model was applied to the same task. 80% of the study samples were randomly sampled to train model parameters. The remaining 20% were used for model validation. Evaluation metrics including precision, recall, and c-statistic were reported.

Results: Both KARA and the baseline achieved high precision (0.984 and 0.951, shown in Table 2) and high recall (0.942 and 0.947) towards non-crisis cases. For crisis cases, however, KARA model achieved a much higher recall than the baseline (0.870 vs 0.791). The c-statistics of KARA and the baseline were 0.815 and 0.760, respectively.

Conclusion: KARA significantly outperformed standard NLP models, demonstrating good translational value and clinical relevance.