An empirical Q-matrix validation method for the polytomous G-DINA model

Abstract

Several empirically based Q-matrix validation methods are available in the literature. However, all the existing methods were developed for cognitive diagnosis models (CDMs) for dichotomous attributes. For some applications, classifying students into more than two categories (e.g., no mastery, basic mastery, and advanced mastery) would be more instructionally relevant. To extend the practical utility of CDMs, methods for validating Q-matrix for CDMs for polytomous attributes are needed. This study focuses on validating the Q-matrix of the generalized deterministic input, noisy “and” gate model for polytomous attributes (pG-DINA model). The pGDI, an extension of the G-DINA model discrimination index (GDI) for polytomous attributes, is proposed. The pGDI serves as the basis of a validation method that can be used not only to identify potential misspecified Qentries, but also suggest more appropriate attribute-level specifications. The following are some of the theoretical properties of the pGDI. First, like the GDI, the correct specification for an item is the smallest q-vector among the q-vectors with the highest pGDI. And second, unique to the pGDI, misspecifying the level of a required attribute leads to a lower discrimination. The practical viability of the proposed method is also examined using a simulation study. Preliminary results show that the method can accurately identify and correct misspecified Q-entries in the pG-DINA model, particularly when high-quality items are involved. Moreover, using the pGDI in conjunction with the mesa plot allows for quantitative and graphical information to be combined, resulting in a more effective implementation of the proposed method.