Generated-versus selected-explanations: results from California and Hong Kong

Abstract

Explanations can be used to examine the extent to which students’ reasoning is based on scientific knowledge and evidence.This study investigates whether and how effectively explanations added to typical multiple choice(MC) items can measure student scientific reasoning.Two types of explanation items were used and compared:generated -explanation(GE ) and selected-explanation(SE ) items.Both explanation items were added to traditional MC items in eliciting student justification of their choices on the MC items.The assessment was administered to 794 Californian students and 916 Hong Kong students.We examined the psychometric properties of the three item types(MC,GE,and SE ) and compared student performance on these three item types between Californian and Hong Kong students.A differential item functioning analysis was conducted to detect items with potential bias.Results show that GE items are more difficult than the other two item types for both Californian and Hong Kong students and tend to favor Californian students.We discuss the performance difference from assessment and curriculum perspectives.

解釋可以用來檢驗學生的推理在多大程度上基于科學知識和證據,本文研究的是學生在兩類經典多選題中的解釋是否可以測量他們的科學推理以及測量的有效性問題。本研究在傳統多選題中附加了兩類解釋項目——生成解釋和選擇解釋,用以了解學生做出選擇的原因。這項研究測試了794個美國加州學生和916個中國香港學生,檢驗了三類題目(傳統多項選擇、生成解釋、選擇解釋)的心理測量學特征,并且比較了美國加州學生和中國香港學生在三類試題上的表現。為了檢測不同國家的潛在偏見,本文還進行了題目功能差異分析。研究結果顯示美國加州學生的分數更高,并且無論對美國加州學生還是對中國香港學生,生成解釋比另外兩類題目難度都要大。本文最后從評估和課程的角度對表現上的差異進行了討論。