Do I complete Q?

Abstract

A central component of most cognitive diagnosis models (CDMs) is the Q-matrix, which specifies the attributes required by each item. A complete Qmatrix is said to be necessary for all the attribute profiles to be identified. However, the completeness, or lack thereof, of a particular Qmatrix may vary from one CDM to another. A method that has been proposed to assess Qmatrix completeness is to compare the success probabilities across the items of the different attribute profiles. However, this method presupposes that the underlying CDMs are known, a condition that is difficult to satisfy in practice. The current work proposes a simulationbased approach to assess Qmatrix completeness. The proposed method involves determining the simplest CDMs empirically, and disentangling completeness from test reliability. A simulation study is conducted to evaluate the viability of the proposed method. Results show that the simulationbased method performs well under most conditions, but needs to be used with caution when the sample size is small and items are of inadequate quality. A set of real data is also analyzed to examine the viability of the proposed procedure.