Identifying poor-fitting items using limited-information statistics for CDM

Abstract

The Pearson χ2 and G2 statistics are the most often used full information statistics for testing goodnessof-fit for latent variable models. However, they are effective only when all the expected frequencies are greater than 5. This requirement becomes difficult to achieve when the number of examinees is relatively smaller than the number of possible response patterns resulting in a sparse response contingency table. Recently, various alternatives using information from the lower margins have been developed (see, e.g., Christofferson, 1975; Bartholomew & Leung, 2002; Reiser, 1996; Maydeu-Olivares & Joe, 2005; Hansen, Cai, Monroe, & Li, 2016). By adopting the limited information model fit indices, this study aims to develop an algorithm that can eliminate poor-fitting items not due to Q-matrix or model misspecification but other reasons e.g. omission of latent attributes within the cognitive diagnosis modeling framework. Two simulations studies are conducted. First, the performance of various limitedinformation statistics such as log odds ratio of item pairs (Chen, de la Torre & Zhang, 2013), Christofferson’s test, Bartholomew and Leung’s test and M2 statistics will be compared with each other and with full information statistics in terms of type I error and power. Next, an algorithm is developed using previously selected methods to eliminate poor-fitting items. Various types of misfit are covered including 1) misspecification of Q-matrix, 2) omission/addition of latent attributes, 3) misspecification of CDMs, and 4) dependency of residuals.