The Challenges of Measuring Outside-School-Time Educational Activities: Experiences and Lessons from the Programme for International Student Assessment (PISA)

Abstract

Recent years have brought a major shift in the field of international comparative education with the rise of quantitative assessments of student achievement. Within these studies, outside-school-time (OST) as a supplement and complement to schooling has been included. However, OST is difficult to define and measure; and challenges multiply when comparisons are made across countries and cultures. The Programme for International Student Assessment (PISA) managed by the Organisation for Economic Co-operation and Development (OECD) has sought to confront these challenges. It has had some success, and the 2015 PISA iteration showed advances over its predecessors; but it also showed significant shortcomings. With a methodological thrust, this paper examines the challenges of preparing appropriate questions in English, translating those questions into other languages, and preparing meaningful cross-national comparisons. The paper has pertinence not only within the specific domain of OST studies but also the wider thrusts of quantification in international comparisons.