What makes listening comprehension difficult?
A feature-based machine learning approach to understanding item difficulty
Bibliographic Data
Understanding what makes second language (L2) listening comprehension difficult is crucial for advancing language learning and assessment. In L2 listening assessment, a key challenge is developing items with targeted difficulty levels. This difficulty can be influenced by textual and acoustic features from different item segments (i.e. stimuli, stems, and options) embedded in a multi-layered structure, along with task-related features. This study explores a feature-based machine learning (ML) approach to predicting difficulty of multiple-choice listening items on a local language proficiency test. We extracted construct-relevant textual and acoustic features from item segments across five dimensions: lexical complexity, syntactic complexity, fluency, pronunciation, and similarities among item segments. Incorporating these features, we compared traditional and mixed-effects ML models for predictive accuracy and interpretability. The best-performing model—a mixed-effects Ridge model with twenty-three features—achieved high accuracy (R2 = 0.860) and showed meaningful feature-difficulty relationships. This study presents methodological innovations for item difficulty modeling and offers practical implications for human- and machine-mediated item development. It also demonstrates potential of incorporating computational linguistics and ML in enhancing L2 listening assessment.
Active listening · Comprehension · Computational linguistics · Key (lock) · Language acquisition · Language proficiency · Listening comprehension · Educational and Psychological Assessments · Language Development and Disorders · Psychometric Methodologies and Testing
Assessing the Validity of Lexical Diversity Indices Using Direct Judgements
Using automated written corrective feedback in the writing classrooms
Suprasegmental Measures of Accentedness and Judgments of Language Learner Proficiency in Oral English
The tool for the automatic analysis of lexical sophistication (Taales)
Exploring task difficulty in ESL listening assessment
Validation of listening comprehension tests
Fairness of using different English accents
To show or not to show
Analyzing Discourse Processing Using a Simple Natural Language Processing Tool
The Effects of Syntactic Simplification and Repetition on Listening Comprehension
The Role of Task and Listener Characteristics in Second Language Listening
Speech Rate and Listening Comprehension
A Theory of Pitch Accent in English
| Citation velocity | historical |
|---|---|
| Highly cited | No |