Evaluation of the consistency of a speech verification system with human raters in early literacy screening assessments
Bibliographic Data
| ID | 22164276 |
|---|---|
| Authors | Yaacov Petscher (0000-0001-8858-3498, Florida State University), Jennifer I O'Sullivan (University of the Republic of San Marino), Jennifer O'Sullivan (Marino Institute of Education), Hugh W Catts (0000-0001-9522-7758, Florida State University), Ashley Edwards (Florida State University), Lisa Fitton (0000-0003-0524-7339, University of South Carolina) |
| Year | 2026 |
| Volume | 11 |
| Publication date | 2026-02-19 |
| Open Access | Yes |
| Type | ARTICLE |
| Venue | Frontiers in Education (JOURNAL) |
| Journal identifiers | ISSN: 2504-284X • E-ISSN: 2504-284X |
| Publisher | Frontiers Media SA (PUBLISHER • CH) |
| DOI | 10.3389/feduc.2026.1671946 |
| OpenAlex | W7130508234 |
| Language | EN |
| References cited | 47 |
This study investigates the use of a speech verification system (SVS) technology, a form of automatic speech recognition (ASR), in the assessment of children's reading skills. Despite the growing integration of ASR systems in educational assessment, significant challenges persist, particularly due to the acoustic, pronunciation and dialectal variability inherent in children's speech. Our research evaluated the consistency between human rater (HR) and speech verification system (SVS) scores produced from SoapBox Labs across three linguistic tasks—phoneme blending, expressive vocabulary and word reading. Results reveal variability in agreement rates, with SVS showing lower consistency with human raters in phonologically complex tasks like phoneme blending, compared to expressive vocabulary and word reading tasks. Additionally, we address potential racial differences in SVS performance, highlighting the importance of diverse speech sample collection to ensure equitable assessments, as well as inter-item differences within a task. The study concludes with recommendations for consideration of using SVS in educational assessments, advocating for ongoing research and algorithmic advancements to better support educational assessment practices
Literacy · Pronunciation · Vocabulary · Language Development and Disorders · Phonetics and Phonology Research · Speech Recognition and Synthesis
Racial disparities in automated speech recognition
The effect of experience on the perception and representation of dialect variants
Acoustics of children’s speech
English with an Accent
Bias in Automatic Speech Recognition
Dialect comprehension and identification in L2 Spanish
The effect of training and rater differences on oral proficiency assessment
How the Science of Reading Informs 21st‐Century Education
Accuracy and Inaccuracy in Teachers’ Perceptions of Young Children’s Cognitive Abilities
Are teachers' expectations different for racial minority than for European American students? A meta-analysis
Misinterpretation of African American English BIN by adult speakers of Standard American English
| Highly cited | No |
|---|