A framework for comparing large-scale survey assessments

Large-scale survey assessments (LSAs) are important tools for measuring educational outcomes and shaping policy decisions. We present a framework for comparing LSAs to facilitate studying the impact of design choice on the precision of results, contrasting India’s National Achievement Survey (NAS), the United States’ National Assessment of Educational Progress (NAEP), and the OECD’s Programme for International Student Assessment (PISA). Our framework focuses on four key elements: sampling design, assessment design, analysis methodology, and reporting. The notion of total survey error, which is the accumulation of errors across the four key elements, can be used for both designing and evaluating LSAs. As example, we compare statistics that are commonly (but not always) reported from NAS, NAEP, and PISA to summarize outcomes related to sampling, measurement, and reporting. Our examination reveals several key similarities and differences among the three assessments, thereby highlighting the nuanced ways in which each LSA is tailored to meet the specific needs of their purpose and the challenges they face

Cartography · Geography · Regional science · Agricultural risk and resilience · Income, Poverty, and Inequality · Social and Economic Development in India

• Test Equating, Scaling, and Linking

  Open Access•Michael J Kolen, Robert L Brennan•Test Equating, Scaling, and Linking•2004

  Incluir item

• Estimating Population Characteristics From Sparse Matrix Samples of Item Responses

  Open Access•Robert J Mislevy, Albert E Beaton et al.•Journal of Educational Measurement•1992

  Incluir item

• The Multidimensional Random Coefficients Multinomial Logit Model

  Open Access•Raymond J Adams, Mark Wilson et al.•Applied Psychological Measurement•1997

  Incluir item

• Randomization-Based Inference about Latent Variables from Complex Samples

  Open Access•Robert J Mislevy•Psychometrika•1991

  Incluir item

• The role of plausible values in large-scale surveys

  Open Access•Margaret Wu•Studies In Educational Evaluation•2005

  Incluir item

• The Total Survey Error Approach

  Herbert F Weisberg•Total Survey Error Approach•2005

  Incluir item

• Multiple Imputation for Nonresponse in Surveys

  Open Access•Donald B Rubin•Multiple Imputation for…•1987

  Incluir item

• The Determination of Sample Size

  Jerome Cornfield•American Journal of Public Health…•1951

  Incluir item

• Total Survey Error

  Paul P Biemer•Public Opinion Quarterly•2010

  Incluir item

• Total Survey Error

  Robert M Groves, Lars Lyberg•Public Opinion Quarterly•2010

  Incluir item