Improving the Validity of Quantitative Measures in Applied Linguistics Research1
Bibliographic Data
| ID | 4932674 |
|---|---|
| Authors | James E Purpura (0009-0007-8642-104X, Teachers College Columbia University, corresponding author), James Dean Brown (0000-0001-9185-3284, University of Hawaiʻi at Mānoa), Rob Schoonen (0000-0002-3500-9262, University of Amsterdam) |
| Year | 2015 |
| Volume | 65 |
| Issue | S1 |
| Pages | 37-75 |
| Publication date | 2015-06-01 |
| Peer Reviewed | Yes |
| Open Access | Yes |
| Type | ARTICLE |
| Venue | Language Learning (JOURNAL) |
| Journal identifiers | ISSN: 0023-8333 • E-ISSN: 1467-9922 |
| Publisher | Wiley (PUBLISHER • GB) |
| DOI | 10.1111/lang.12112 |
| OpenAlex | W1625029858 |
| Language | EN |
| Citations received | 19 |
| References cited | 39 |
In empirical applied linguistics research it is essential that the key variables are operationalized in a valid and reliable way, and that the scores are treated appropriately, allowing for a proper testing of the hypotheses under investigation. The current article addresses several theoretical and practical issues regarding the use of measurement instruments and scores. Starting from an elaborated treatment of validity and validation, it introduces a comprehensive framework for score interpretation and score use. Kane's framework serves as a rich context to raise the right questions about a measurement instruments' validity, and it provides guidance in addressing questions about validity and score use. Application of the framework is illustrated with examples from a recent second language acquisition study about the effects of recasts (Révész, ). The critical generalization inference from the framework, as it applies to measuring productive L2 performance, is then examined in greater detail, discussing the different facets of these kinds of measurements and the way reliability issues are and should be dealt with. The paper concludes with a series of common measurement mistakes in applied linguistics research and recommendations to avoid these. The paper offers a brief checklist for proper quantitative data collection and for adequate data treatment in subsequent analyses
Applied linguistics · Checklist · Cognitive psychology · Context (archaeology) · Epistemology · External validity · Generalization · Interpretation (philosophy) · Linguistics · Management science · Operationalization · Psychometrics · Reliability (semiconductor) · Test validity · Validity · Computer Science · EFL/ESL Teaching and Learning · Language, Discourse, Communication Strategies · Multilingual Education and Policy · Psychology · Social Psychology
Scale quality in second-language anxiety and WTC
Comparing the longitudinal development of phraseological complexity across oral and written tasks
Resolving the crisis in L2 motivational self-system research
Applied linguistics research in three decades
The development and initial validation of LexCombi 2
Methodological orientations, academic citations, and scientific collaboration in applied linguistics
Developing internet-based Tests of Aptitude for Language Learning (Tall)
Managing proposal sequences in role-play assessment
A Context-Aligned Two Thousand Test
Open Science practices in language assessment
Preparing for the higher education admission test
Tesol Quarterly Research Guidelines
Effects of different forms of explicit instruction on L2 development
Statistical Significance Testing in Second Language Research
Design Issues and Inference in Experimental L2 Research
Measuring Foreign Language Students' Self-Determination
Probing the Internal Validity of the LLAMA Language Aptitude Tests
Proficiency Reporting Practices in Research on Second Language Acquisition
Study and Scale Quality in Second Language Survey Research, 2009-2019
The Encyclopedia of Applied Linguistics
Assessing Writing
Statistical Analyses for Language Assessment
Assessing Speaking
Applying the Rasch Model
The Uses of Argument
Generalizability Theory
Best Practices in Quantitative Methods
Building and Supporting a Case for Test Use
Language Proficiency in Native and Nonnative Speakers
Study Quality in Sla
Focus Article
Validating the Interpretations and Uses of Test Scores
Magnitude of Task-Sampling Variability in Performance Assessment
Dependability of scores for a new ESL speaking assessment consisting of integrated and independent tasks
Design and analysis in task-based language assessment
Second language interaction
An introduction to structural equation modelling for language assessment research
Score generalizability of academic writing tasks
Generalizability of writing scores
Validity evidence in a university group oral test
Task‐Based Language Learning
Cultures de production, de distribution et de consommation en milieu urbain en Angleterre, 1100-1350
Statistics as a Foreign Language
Working Memory and the Observed Effectiveness of Recasts on Different L2 Outcome Measures
Testing Pronunciation
Measures of Linguistic Accuracy in Second Language Writing Research
Effectiveness of L2 Instruction
An argument-based approach to validity
| Unique citing works | 19 |
|---|---|
| Citations per year | 1,73 |
| Citation span | 2015 - 2026 (12) |
| Citation velocity | current |
| Highly cited | No |
| Citation types | Neutral: 19 |