Examining the relevance of three Toefl Essentials writing tasks to the accounting profession

Large-scale English language proficiency tests are increasingly used to make decisions about professional registration despite not originally being developed to make predictions about language use in the workplace. Domain experts can play a valuable role as informants in establishing the relevance of test tasks to a specific TLU domain. However, this practice has seldom been critically examined. In particular, no studies to date have examined the nature of the interview questions used when engaging with domain experts in this type of research. The current study was designed with two aims: to (1) explore the relevance of three writing tasks (i.e., Build a Sentence, Write for an Academic Discussion, Write an Email) from the TOEFL Essentials test to the accounting profession and (2) evaluate the judgements of domain expert participants. Twenty accountants from non-English speaking backgrounds as well as three accounting educators were interviewed for the study, drawing on a methodology with broad, general questions. The data was analysed qualitatively to identify (a) to what extent the participants considered the three tasks relevant and (b) what task features they attended to when commenting on the relevance of the tasks. The findings showed that the participants generally found the Build a Sentence the least relevant of the three writing tasks, and the Write an Email task the most relevant. When reviewing the task features the participants judged as relevant to writing demands in their workplace, it was shown they focused on a small/narrow range of features. They also engaged in ‘misinterpretations’, comparing aspects of test and workplace tasks that did not align (e.g., comparing a writing task to events in a spoken meeting). The findings are discussed in terms of domain expert involvement in validation research.

Language proficiency · Sentence · Subject-matter expert · Test of English as a Foreign Language · Accounting Education and Careers · EFL/ESL Teaching and Learning · Student Assessment and Feedback

• What Counts in Second Language Oral Communication Ability? The Perspective of Linguistic Laypersons

  Open Access•Takanori Sato, Tim Mcnamara•Applied Linguistics•2019

  Incluir item

• Validity and washback in language testing

  Open Access•Samuel Messick•Language Testing•1996

  Incluir item

• The hunt for “indigenous criteria” in assessing communication in the physiotherapy workplace

  Open Access•Catherine Elder, Tim Mcnamara•Language Testing•2016

  Incluir item

• The Interplay of Evidence and Consequences in the Validation of Performance Assessments

  Samuel Messick•Educational Researcher•1994

  Incluir item

• Interrogating the construct of communicative competence in language assessment contexts

  Open Access•Catherine Elder, Tim Mcnamara et al.•Language & Communication•2017

  Incluir item