Prioritising authenticity in live lecture comprehension pre-sessional assessment task design
Bibliographic Data
| ID | 7159167 |
|---|---|
| Authors | Stefan O'Grady (0000-0003-3810-713X, University of St Andrews, corresponding author), Aaron Rotsinger (University of St Andrews), Mark Carver (0000-0003-4393-8915, University of St Andrews) |
| Year | 2025 |
| Volume | 77 |
| Pages | 101554 |
| Publication date | 2025-09-01 |
| Peer Reviewed | Yes |
| Open Access | Yes |
| Type | ARTICLE |
| Venue | Journal of English for Academic Purposes (JOURNAL) |
| Journal identifiers | ISSN: 1475-1585 • E-ISSN: 1878-1497 |
| Publisher | Elsevier BV (PUBLISHER) |
| DOI | 10.1016/j.jeap.2025.101554 |
| OpenAlex | W4412494193 |
| Language | EN |
| References cited | 30 |
Where pre-sessional assessments determine student progression, authenticity is crucial to deliver construct validity and support high-stakes progress decisions. However, replicating the natural processes that occur in academic environments poses challenges relating to domain representation and test-taker engagement as well as meeting stakeholder needs such as benchmarking against commercial tests. The current study reports on a project devised to develop an authentic lecture assessment for a pre-sessional programme, informed by a university-wide needs analysis. In the study, 66 students completed a live lecture comprehension task, and data were analysed using classical test theory and many facet Rasch measurement. In addition, students completed a questionnaire which was analysed using stepwise linear regression. Analysis of the lecture transcript was completed to evaluate the suitability of the content for the target level. Results showed that speech rate and vocabulary were appropriate for the assessment objectives, student self-reported comprehension accounted for some test score variance, and the test was reliable. However, many facet Rasch measurement showed the test was only able to categorise test takers into two distinct levels, which was problematic as the pre-sessional needed to multiple levels for schools to make admissions decisions. Recommendations for balancing these challenges in assessment design are presented
Comprehension · Mathematics education · Programming language · Computer Science · Educational Strategies and Epistemologies · Engineering · Innovative Teaching and Learning Methods · Psychology · Student Assessment and Feedback
The Routledge Handbook of Research Methods in Applied Linguistics
MTLD, vocd-D, and HD-D
Multiple Regression in L2 Research
Topic and Accent Coverage in a Commercialized L2 Listening Test
The impact of input format on written performance in a listening-into-writing assessment
Notetaking as validity evidence
Speech production and second language acquisition
Furthering the argument for visually inclusive L2 academic listening tests
The typology of second language listening constructs
An eye-tracking study of attention to visual cues in L2 listening tests
The effects of test anxiety on listening test performance
Investigating differences in test-takers’ use of cognitive and metacognitive strategies in audio-only and video-based listening comprehension test
Multimodal competence and effective interactive lecturing
The Role of Task and Listener Characteristics in Second Language Listening
Repeating the Listening Text
Student question types and subject-position pronominal choices
| Citation velocity | historical |
|---|---|
| Highly cited | No |