A cognitive diagnosis model analysis of a digital literacy assessment

Abstract

Digital literacy is a transversal competency needed to be successful in this technology-intensive society. To investigate the digital literacy of Hong Kong students, an assessment measuring five digital skills, namely, information and data literacy (A1), communication and collaboration (A2), digital content creation (A3), safety (A4), and problem solving (A5), was developed. We propose to use a general cognitive diagnosis model framework to examine the mastery of digital literacy skills of Hong Kong primary students. In addition, the relationship between digital skill mastery and a number of demographic variables is also investigated. This study analyzes data collected from 422 Grade 3 students. To determine mastery of various digital skills and test properties, the G-DINA model framework is fitted to the data; to determine the relationship between skill mastery and covariates, the former is regressed on the latter. Preliminary results indicate that several items with low discrimination may need to be revisited to determine whether modifying the item-skill specifications may improve the item quality. Notwithstanding the current test quality, students’ skill mastery can be accurately classified – A1 and A4 have the highest attribute classification accuracy (.93), whereas A3 has the lowest (.77). The results also show that all digital literacy skills have low mastery proportions, with A1 and A4 having the lowest (.25) and highest (.43) proportions of mastery, respectively. Finally, the results indicate that language and digital device access are differentially related to the mastery of digital literacy skills, whereas gender and school religion are not.