General CDM joint attribute model formulation and selection

Abstract

Cognitive diagnosis models (CDMs) provide finer-grained information that can be used to foster teaching and learning. However, when CDMs measure a large number of attributes, or when the testing condition is less than ideal (e.g., short tests or small number of examinees are involved), the attribute classification accuracy can suffer when the attribute joint distribution is formulated generally. With the goal of identifying the best model in non-ideal conditions, the joint attribute model with higher-order (HO) latent trait structure is used with the generalized deterministic input, noisy “and” gate model (G-DINA) model. The HO model assumes that attribute mastery is governed by a smaller number of HO abilities (typically, one), and that the attributes are conditionally independent given the abilities. To investigate the robustness of the HO and G-DINA model combination, three factors, namely, joint attribute distribution (uniform, independence, one-dimensional HO, and two-dimensional HO), item quality (low, medium, and high), and test length are examined. The HO model is compared with saturated G-DINA model in terms of the attribute classification accuracy and model fit. Preliminary results show that the HO model performs better than the saturated model across the different attribute distributions, where more obvious differences can be found in the lower item quality conditions. These findings suggest that the HO model provides a flexible and robust parametrization of the joint attribute distribution in the G-DINA model context. Guidelines on how relative and absolute fit statistics can be used in choosing between the HO and saturated models are also discussed.