Attribute Classification Accuracy Improvement: Monotonicity Constraints on the G-DINA Model

Abstract

Cognitive Diagnosis Models (CDMs) are restricted latent class models developed to identify students’ mastery and nonmastery of multiple attributes. A common indicator of reliability in CDM is Attribute Classification Accuracy (ACA). In this work, we explore the consequences of assuming an inappropriate model, and propose a new version of a general CDM, G-DINA, where a monotonic constraint is included. A simulation study is conducted to investigate how the ACA of monotonic G-DINA compares with those of G-DINA and other reduced CDMs. The comparison involves both calibration and validation samples. We also introduce the use of the Likelihood Ratio (LR) test to evaluate the appropriateness of imposing this nonlinear constraint. LR Type I error and power in this context is evaluated. For comparison purposes, the performance of AIC and BIC is also documented. Results show that the ACA of the monotonic G-DINA model is always better than that of the G-DINA model, and approaches that of the generating reduced CDMs. These differences were more pronounced in the validation sample indicating that the lack of parsimony of the G-DINA model affects the generalizability and suitability of the item parameter estimates across samples. The results also show that the LR test can be used to determine whether or not monotonicity can be assumed. Overall, this study finds that the appropriateness of the constrained version of the G-DINA model can be tested empirically, and its proper use (i.e., in situations where the true CDMs cannot be assumed) leads to improved ACA.