Automatic Coding of Short Text Responses via Clustering in Educational Assessment

Dados Bibliográficos

ID20282974
AutoresFabian Zehner (0000-0003-3512-1403, Technische Universität München, Munich, Germany, autor correspondente), Christine Sälzer (0000-0002-8064-4708, Technische Universität München, Munich, Germany), Frank Goldhammer (0000-0003-0289-9534, Centre for International Student Assessment (ZIB) e.V., Munich, Frankfurt am Main, Kiel, Germany)
Ano2016
Volume76
Fascículo2
Páginas280-303
Data de publicação2016-04-01
Peer ReviewedSim
Open AccessSim
TipoARTICLE
PeriódicoEducational and Psychological Measurement (JOURNAL)
Identificadores do periódicoISSN: 0013-1644 • E-ISSN: 1552-3888
EditoraSAGE Publications (PUBLISHER • US)
DOI10.1177/0013164415590022
PMID29795866
OpenAlexW1848614604
IdiomaEN
Citações recebidas10
Referências citadas24

Automatic coding of short text responses opens new doors in assessment. We implemented and integrated baseline methods of natural language processing and statistical modelling by means of software components that are available under open licenses. The accuracy of automatic text coding is demonstrated by using data collected in the Programme for International Student Assessment (PISA) 2012 in Germany. Free text responses of 10 items with [Formula: see text] responses in total were analyzed. We further examined the effect of different methods, parameter values, and sample sizes on performance of the implemented system. The system reached fair to good up to excellent agreement with human codings [Formula: see text] Especially items that are solved by naming specific semantic concepts appeared properly coded. The system performed equally well with [Formula: see text] and somewhat poorer but still acceptable down to [Formula: see text] Based on our findings, we discuss potential innovations for assessment that are enabled by automatic coding of short text responses

Cluster analysis · Coding (social sciences) · Natural language processing · Statistics · Text messaging · Artificial Intelligence · Computer Science · Mathematics · Natural Language Processing Techniques · Text Readability and Simplification · Topic Modeling