Confidence-aware truth estimation in social sensing applications

Abstract

This paper presents a confidence-aware maximum likelihood estimation framework to solve the truth estimation problem in social sensing applications. Social sensing has emerged as a new paradigm of data collection, where a group of individuals volunteer (or are recruited) to share certain observations or measurements about the physical world. A key challenge in social sensing applications lies in ascertaining the correctness of reported observations from unvetted data sources with unknown reliability. We refer to this problem as truth estimation. The prior works have made significant efforts to solve this problem by developing various truth estimation algorithms. However, an important limitation exists: they assumed a data source makes all her/his observations with the same degree of confidence, which may not hold in many real-world social sensing applications. In this paper, we develop a new confidence-aware truth estimation scheme that removes this limitation by explicitly considering different degrees of confidence that sources express on the reported data. The new truth estimation scheme solves a maximum likelihood estimation problem to determine both the correctness of collected data and the reliability of data sources. We compare our confidence-aware scheme with the state-of-the-art baselines through both an extensive simulation study and three real world case studies based on Twitter. The evaluation shows that our new scheme outperforms all compared baselines and significantly improves the accuracy of the truth estimation results in social sensing applications.