Relating Hong Kong students' digital literacy and background characteristics using a cognitive diagnosis modeling framework

Abstract

Digital literacy (DL), which is one of the critical transversal competencies needed in the current technology-intensive society, has received increasing attention from educational researchers and practitioners in recent years. In light of the importance of DL, a number of DL assessments have been developed to assess students’ DL achievement. However, most existing studies summarize students' DL achievement using traditional methods or item response theory models, neglecting the importance of diagnostic information that can be used for remediation and classroom instruction for formative assessment purposes. In addition, although the relationships between students’ overall DL achievement and their background characteristics have been extensively studied, skill-level relationships remain unsolved. Moreover, the existing body of DL literature has mainly focused on students from a single age and has seldom compared students’ DL achievements between age cohorts. Using the data collected by the digital literacy assessment (DLA) and the student survey, which are part of a larger Theme-based Research Scheme project called Learning and Assessment for Digital Citizenship in Hong Kong, this thesis aims to measure and compare Hong Kong students' strengths and weaknesses on five DL skills, and investigate related background characteristics, using a general cognitive diagnosis model (CDM) framework. The sample consists of students from three age cohorts (i.e., Primary 3, Secondary 1, and Secondary 3). Study 1 analyzes the DLA data using the multiple-group generalized deterministic inputs, noisy “and” gate (MG-GDINA) model. Acceptable model-data fit results provide evidence for the viability and utility of CDM analysis. The findings indicate that secondary students had better mastery of all five DL skills than primary students, and reveal different patterns of DL mastery profiles among the three cohorts. Study 2 investigates whether gender and SES are related to students’ mastery of digital skills across the three age cohorts using latent logistic regression. Our results show that gender differences in favor of females only exist at Secondary 1 and 3, but not at Primary 3, whereas, SES is positively related to all digital skills in all three age cohorts. The significant interaction between gender and SES at Secondary 1 indicates that SES is only related to girls’ DL mastery for this age cohort. To provide an appropriate analytical approach for further longitudinal investigation of DL and its related factors at the skill level, Study 3 proposes a bias-corrected three-step estimation approach for latent transition CDM with covariates. The proposed approach allows researchers to assess students’ changes in attribute mastery and evaluate covariate effects on both the initial state and transition probabilities over time. Results show that the proposed method yields more accurate parameter estimates than the uncorrected three-step approach. In summary, Hong Kong students’ DL mastery and the related background characteristics are investigated and compared between different age cohorts in this thesis. Findings from this thesis provide insights into Hong Kong students’ mastery of digital skills and the relationships between DL mastery and gender and SES in the local context. Finally, this thesis also contributes methodologically to the CDM analysis of longitudinal assessments data.