Xun Yan
Biographic Data
| ID | 3843741 |
|---|---|
| NAME | Xun Yan |
| GIVEN NAMES | Xun |
| FAMILY NAME | Yan |
| SIGNATURE | YAN X |
| AFFILIATIONS | University of Illinois Urbana-Champaign |
| ORCID | 0000-0002-4544-4938 |
| VERIFIED | Yes |
| TOTAL WORKS | 35 |
| TOTAL CITATIONS | 10 |
| AUTHOR COUNT | 35 |
| EDITOR COUNT | 0 |
| FIRST PUBLICATION YEAR | 2014 |
| LATEST PUBLICATION YEAR | 2026 |
| H-INDEX | 2 |
Writing assessment literacy of teachers of languages other than English: A multi-case study
Considering the predominance of research on writing assessment literacy (WAL) in English contexts, this multiple-case study investigates the WAL of tertiary-level teachers of languages other than English (LOTE) in China. Five LOTE teachers across German, Russian, Korean, and Chinese participated in in-depth interviews and provided assessment-related materials. The findings revealed variability in their WAL across participants and imbalances withi…
Expanding horizons in Language Testing: Policies, dialogues, and new directions
Assessment of Fluency
Second language (L2) speech fluency assessment has gained significant attention in the field of language acquisition and education. Fluency plays a crucial role in effective communication and language proficiency. As a multidimensional construct subsumed under oral language proficiency, fluency has been conceptualized in both broad and narrow senses. However, in terms of operationalization, fluency has been mostly examined through temporal charac…
What makes listening comprehension difficult?: A feature-based machine learning approach to understanding item difficulty
Understanding what makes second language (L2) listening comprehension difficult is crucial for advancing language learning and assessment. In L2 listening assessment, a key challenge is developing items with targeted difficulty levels. This difficulty can be influenced by textual and acoustic features from different item segments (i.e. stimuli, stems, and options) embedded in a multi-layered structure, along with task-related features. This study…
Disfluency doesn’t happen in isolation: Exploring how individual disfluency features co-occur in L2 speaking performances
The construct of second language (L2) utterance fluency is typically operationalized through various individual temporal features. However, in natural speech, fluency (or disfluency) is often characterized by the clustering of multiple temporal features, collectively revealing the speaker’s effort in speech production or disfluency recovery. In this study, we explore the co-occurrence patterns of disfluency features in L2 speech and their associa…
Language Testing behind the scenes: New capabilities, groundbreaking developments, and disruptive forces
Translingual practices of foreign friendship talks in ELF: A discourse-analytic approach
Language assessment in the era of generative artificial intelligence: Opportunities, challenges, and future directions
Generative AI for the Teaching, Learning, and Assessment of Productive Skills: An Evidence‐Based Approach to Understanding Its Real Impact
Generative Artificial Intelligence in English Language Education: Potential, Challenges, and the Path Forward
Diving Deep Into the Relationship Between Speech Fluency and Second Language Proficiency: A Meta-Analysis
Abundant research has indicated fluency features as meaningful predictors of second language proficiency. However, the extent to which different fluency dimensions and features can predict proficiency remains underexplored. This meta-analysis employed a multilevel modeling approach to synthesize fluency-proficiency relationships from 71 empirical studies from 1959-2023. Additionally, we examined several moderator variables, including task type, l…
Incorporating Co-occurrence Into the Operationalization of Speech Disfluency for Second Language Pronunciation and Oral Proficiency Assessment
Current L2 utterance fluency literature tends to operationalize disfluency as isolated, individual features. However, disfluency features often co-occur at one location or across multiple locations in one utterance. This study explores the co-occurrence of L2 disfluency features in a speech corpus from 71 L1 and L2 speakers of English across proficiency levels on an elicited imitation task and an oral listen-to-summarize task. We segmented each p…
Are Textbooks Authentic From a Developmental Perspective? A Corpus Analysis of Word Use in Chinese Textbooks in US Universities
There is a prevailing belief that second language (L2) textbooks should strive for authenticity, aiming to accurately and reliably represent natural language use. However, assessing the authenticity of language textbooks is not a straightforward task, as it requires both a comparison between instructional texts and real-world language use and a consideration of the comparison within the broader context of language development. This study employs …
Economic return of English language proficiency: Do English proficiency scores predict employee income in China
As English is the international language of the global economy, possessing sufficient English language skills or passing large-scale standardized proficiency tests is believed to have various economic returns. Situating this common belief about English proficiency test scores within the argument-based validity framework, economic returns of English proficiency can be considered a source of validity evidence at the extrapolation inference level. U…
Fairness of using different English accents: The effect of shared L1s in listening tasks of the Duolingo English test
This study aimed to answer an ongoing validity question related to the use of nonstandard English accents in international tests of English proficiency and associated issues of test fairness. More specifically, we examined (1) the extent to which different or shared English accents had an impact on listeners’ performances on the Duolingo listening tests and (2) the extent to which different English accents affected listeners’ performances on two …
Triangulating natural language processing (NLP)-based analysis of rater comments and many-facet Rasch measurement (MFRM): An innovative approach to investigating raters’ application of rating scales i…
Rater comments tend to be qualitatively analyzed to indicate raters’ application of rating scales. This study applied natural language processing (NLP) techniques to quantify meaningful, behavioral information from a corpus of rater comments and triangulated that information with a many-facet Rasch measurement (MFRM) analysis of rater scores. The data consisted of ratings on 987 essays by 36 raters (a total of 3948 analytic scores and 1974 rater …
Physical exercise and anxiety: The chain mediating effect of social support and resilience
We explored the association between college students’ physical exercise and anxiety by analyzing the mediating role of social support and resilience, and their chain mediation effect. We collected data from 325 university students, who completed the Physical Activity Readiness Scale, the Self-Rating Anxiety Scale, the Social Support Rating Scale, and the Connor–Davidson Resilience Scale. The research findings indicated that physical exercise had …
Connecting source use and argumentation in L2 integrated argumentative writing performance
How do raters learn to rate? Many-facet Rasch modeling of rater performance over the course of a rater certification program
This study employed a mixed-methods approach to examine how rater performance develops during a semester-long rater certification program for an English as a Second Language (ESL) writing placement test at a large US university. From 2016 to 2018, we tracked three groups of novice raters ( n = 30) across four rounds in the certification program. Using many-facet Rasch modeling, rater performance was examined in terms of rater agreement, rater con…
An investigation of the relationship between argument structure and essay quality in assessed writing
Picking Losers: How Career Incentives Undermine Industrial Policy in Chinese Cities
Rapid economic growth in China with considerable market intervention has led to renewed interest in the efficacy of industrial policy. Have industrial policies helped Chinese firms overcome market failures and boosted productivity, or have they distorted markets and undermined creative destruction? This paper explores these questions in the specific context of local implementation by studying the targeting of financial favors and the relationship…
Local tests, local contexts
Dimensionality of speech fluency: Examining the relationships among complexity, accuracy, and fluency (CAF) features of speaking performances on the Aptis test
Speech fluency has been extensively researched as a core construct for second language (L2) speaking assessment. Despite the broad consensus on its multifaceted nature, few researchers have empirically explored the dimensionality of this construct. Operationalizations of fluency vary across research and practice, using both holistic and fine-grained features. To address the dimensionality of speech fluency, in this study we examined an array of f…
“Am I qualified to be a language tester?”: Understanding the development of language assessment literacy across three stakeholder groups
Recent investigations into language assessment literacy (LAL) suggest that stakeholder groups might differ in interests, needs, and expectations in assessment practice, resulting in different LAL profiles. This qualitative study furthers this line of research by examining the effect of contextual and experiential factors on the LAL profiles and development of different stakeholder groups. Semi-structured interviews were conducted among 20 partici…
Using community of practice to characterize collaborative essay prompt writing and its role in developing language assessment literacy for pre-service language teachers
Unpacking the Relationship Between Formulaic Sequences and Speech Fluency on Elicited Imitation Tasks: Proficiency Level, Sentence Length, and Fluency Dimensions
Previous research has provided ample evidence for the processing advantage of formulaic sequences, leading researchers and teachers to argue for its facilitation of speech fluency. However, few studies have examined how the processing of formulaic sequences interacts with speaker proficiency and task difficulty; even fewer studies have investigated how formulaic sequences facilitate different dimensions of speech fluency. To address these gaps, t…
A corpus‐driven, curriculum‐based Chinese elicited imitation test in US universities
This two‐phased study explored a corpus‐driven, curriculum‐based approach to assessing proficiency development of Mandarin Chinese in US universities. In Phase I, we developed an elicited imitation (EI) test based on the Chinese curriculum at a large US university. This test demonstrated higher psychometric qualities to (1) place L2 learners into three curricular levels, and (2) assess learners' production of key vocabulary and grammatical items,…
Using community of practice to characterize collaborative essay prompt writing and its role in developing language assessment literacy for pre-service language teachers
Diving Deep Into the Relationship Between Speech Fluency and Second Language Proficiency: A Meta-Analysis
Abundant research has indicated fluency features as meaningful predictors of second language proficiency. However, the extent to which different fluency dimensions and features can predict proficiency remains underexplored. This meta-analysis employed a multilevel modeling approach to synthesize fluency-proficiency relationships from 71 empirical studies from 1959-2023. Additionally, we examined several moderator variables, including task type, l…
An examination of rater performance on a local oral English proficiency test: A mixed-methods approach
This paper reports on a mixed-methods approach to evaluate rater performance on a local oral English proficiency test. Three types of reliability estimates were reported to examine rater performance from different perspectives. Quantitative results were also triangulated with qualitative rater comments to arrive at a more representative picture of rater performance and to inform rater training. Specifically, both quantitative (6338 valid rating s…
Elicited imitation as a measure of second language proficiency: A narrative review and meta-analysis
Elicited imitation (EI) has been widely used to examine second language (L2) proficiency and development and was an especially popular method in the 1970s and early 1980s. However, as the field embraced more communicative approaches to both instruction and assessment, the use of EI diminished, and the construct-related validity of EI scores as a representation of language proficiency was called into question. Current uses of EI, while not discoun…
Comparing Groups with Multiple Independent Variables
This chapter provides a practical introduction to factorial analysis of variance (ANOVA), a statistical technique widely used in applied linguistics to examine the impact of multiple categorical independent variables on a continuous dependent variable. Focusing primarily on between‐subjects designs, the chapter introduces key concepts such as main effects, interaction effects, and model assumptions including normality, homogeneity of variance, an…
Interpreting the relationships between Toefl iBT scores and GPA: Language proficiency, policy, and profiles
This study examines the predictive validity of the TOEFL iBT with respect to academic achievement as measured by the first-year grade point average (GPA) of Chinese students at Purdue University, a large, public, Research I institution in Indiana, USA. Correlations between GPA, TOEFL iBT total and subsection scores were examined on 1990 mainland Chinese students enrolled across three academic years (N 2011 = 740, N 2012 = 554, N 2013 = 696). Subs…
“Assessment knowledge is important, but …”: How contextual and experiential factors mediate assessment practice and training needs of language teachers
Factor analysis for fairness: Examining the impact of task type and examinee L1 background on scores of an ITA speaking test
This study investigated the construct validity of a local speaking test for international teaching assistants (ITAs) from a fairness perspective, by employing a multi-group confirmatory factor analysis (CFA) to examine the impact of task type and examinee first language (L1) background on the internal structure of the test. The test consists of three types of integrated speaking tasks (i.e., text-speaking, graph-speaking, and listening-speaking) …
Diagnosing English reading ability in Chinese senior high schools
Fitting MD analysis in an argument-based validity framework for writing assessment: Explanation and generalization inferences for the Ecpe
The argument-based approach to validity (Kane, 2013) focuses on two steps: (1) making claims about the proposed interpretation and use of test scores as a coherent, interpretive argument; and (2) evaluating those claims based on theoretical and empirical evidence related to test performances and scores. This paper discusses the role of multidimensional (MD) analysis (Biber, 1988), a corpus analytic approach to examining linguistic and discourse c…
Unpacking the Relationship Between Formulaic Sequences and Speech Fluency on Elicited Imitation Tasks: Proficiency Level, Sentence Length, and Fluency Dimensions
Previous research has provided ample evidence for the processing advantage of formulaic sequences, leading researchers and teachers to argue for its facilitation of speech fluency. However, few studies have examined how the processing of formulaic sequences interacts with speaker proficiency and task difficulty; even fewer studies have investigated how formulaic sequences facilitate different dimensions of speech fluency. To address these gaps, t…
A corpus‐driven, curriculum‐based Chinese elicited imitation test in US universities
This two‐phased study explored a corpus‐driven, curriculum‐based approach to assessing proficiency development of Mandarin Chinese in US universities. In Phase I, we developed an elicited imitation (EI) test based on the Chinese curriculum at a large US university. This test demonstrated higher psychometric qualities to (1) place L2 learners into three curricular levels, and (2) assess learners' production of key vocabulary and grammatical items,…
Dimensionality of speech fluency: Examining the relationships among complexity, accuracy, and fluency (CAF) features of speaking performances on the Aptis test
Speech fluency has been extensively researched as a core construct for second language (L2) speaking assessment. Despite the broad consensus on its multifaceted nature, few researchers have empirically explored the dimensionality of this construct. Operationalizations of fluency vary across research and practice, using both holistic and fine-grained features. To address the dimensionality of speech fluency, in this study we examined an array of f…
“Am I qualified to be a language tester?”: Understanding the development of language assessment literacy across three stakeholder groups
Recent investigations into language assessment literacy (LAL) suggest that stakeholder groups might differ in interests, needs, and expectations in assessment practice, resulting in different LAL profiles. This qualitative study furthers this line of research by examining the effect of contextual and experiential factors on the LAL profiles and development of different stakeholder groups. Semi-structured interviews were conducted among 20 partici…
Using community of practice to characterize collaborative essay prompt writing and its role in developing language assessment literacy for pre-service language teachers
An investigation of the relationship between argument structure and essay quality in assessed writing
Picking Losers: How Career Incentives Undermine Industrial Policy in Chinese Cities
Rapid economic growth in China with considerable market intervention has led to renewed interest in the efficacy of industrial policy. Have industrial policies helped Chinese firms overcome market failures and boosted productivity, or have they distorted markets and undermined creative destruction? This paper explores these questions in the specific context of local implementation by studying the targeting of financial favors and the relationship…
Local tests, local contexts
Connecting source use and argumentation in L2 integrated argumentative writing performance
How do raters learn to rate? Many-facet Rasch modeling of rater performance over the course of a rater certification program
This study employed a mixed-methods approach to examine how rater performance develops during a semester-long rater certification program for an English as a Second Language (ESL) writing placement test at a large US university. From 2016 to 2018, we tracked three groups of novice raters ( n = 30) across four rounds in the certification program. Using many-facet Rasch modeling, rater performance was examined in terms of rater agreement, rater con…
Are Textbooks Authentic From a Developmental Perspective? A Corpus Analysis of Word Use in Chinese Textbooks in US Universities
There is a prevailing belief that second language (L2) textbooks should strive for authenticity, aiming to accurately and reliably represent natural language use. However, assessing the authenticity of language textbooks is not a straightforward task, as it requires both a comparison between instructional texts and real-world language use and a consideration of the comparison within the broader context of language development. This study employs …
Economic return of English language proficiency: Do English proficiency scores predict employee income in China
As English is the international language of the global economy, possessing sufficient English language skills or passing large-scale standardized proficiency tests is believed to have various economic returns. Situating this common belief about English proficiency test scores within the argument-based validity framework, economic returns of English proficiency can be considered a source of validity evidence at the extrapolation inference level. U…
Fairness of using different English accents: The effect of shared L1s in listening tasks of the Duolingo English test
This study aimed to answer an ongoing validity question related to the use of nonstandard English accents in international tests of English proficiency and associated issues of test fairness. More specifically, we examined (1) the extent to which different or shared English accents had an impact on listeners’ performances on the Duolingo listening tests and (2) the extent to which different English accents affected listeners’ performances on two …
Triangulating natural language processing (NLP)-based analysis of rater comments and many-facet Rasch measurement (MFRM): An innovative approach to investigating raters’ application of rating scales i…
Rater comments tend to be qualitatively analyzed to indicate raters’ application of rating scales. This study applied natural language processing (NLP) techniques to quantify meaningful, behavioral information from a corpus of rater comments and triangulated that information with a many-facet Rasch measurement (MFRM) analysis of rater scores. The data consisted of ratings on 987 essays by 36 raters (a total of 3948 analytic scores and 1974 rater …
Physical exercise and anxiety: The chain mediating effect of social support and resilience
We explored the association between college students’ physical exercise and anxiety by analyzing the mediating role of social support and resilience, and their chain mediation effect. We collected data from 325 university students, who completed the Physical Activity Readiness Scale, the Self-Rating Anxiety Scale, the Social Support Rating Scale, and the Connor–Davidson Resilience Scale. The research findings indicated that physical exercise had …
Assessment of Fluency
Second language (L2) speech fluency assessment has gained significant attention in the field of language acquisition and education. Fluency plays a crucial role in effective communication and language proficiency. As a multidimensional construct subsumed under oral language proficiency, fluency has been conceptualized in both broad and narrow senses. However, in terms of operationalization, fluency has been mostly examined through temporal charac…
What makes listening comprehension difficult?: A feature-based machine learning approach to understanding item difficulty
Understanding what makes second language (L2) listening comprehension difficult is crucial for advancing language learning and assessment. In L2 listening assessment, a key challenge is developing items with targeted difficulty levels. This difficulty can be influenced by textual and acoustic features from different item segments (i.e. stimuli, stems, and options) embedded in a multi-layered structure, along with task-related features. This study…
Psychology (23 works) · Linguistics (17 works) · EFL/ESL Teaching and Learning (15 works) · Mathematics education (14 works) · Second Language Learning and Teaching (10 works) · Computer Science (9 works) · Language proficiency (9 works) · Student Assessment and Feedback (8 works) · Second Language Acquisition and Learning (7 works) · Cognitive psychology (5 works)