Proper Sample Sizes for English Language Testing

A Simple Statistical Analysis

Bibliographic Data

ID22129555
AuthorsFaisal Mustafa (0000-0001-7861-0712, Universitas Syiah Kuala), Roderick Julian Robillos (0000-0001-7190-7679, Khon Kaen University)
Year2020
Volume8
Issue4
Pages442-452
Publication date2020-08-13
Open AccessYes
TypeARTICLE
VenueHumanities & Social Sciences Reviews (JOURNAL)
Journal identifiersISSN: 2395-6518 • E-ISSN: 2395-6518
PublisherMaya Global Education Society (PUBLISHER)
DOI10.18510/hssr.2020.8443
OpenAlexW3049240108
LanguageEN
Citations received1
References cited28

Purpose of study: Small sample size is the most common limitation which restricts the generalization of research results, and this is true to many fields, including language testing. The current study is sought to show the predictive power of sample sizes over the population mean to decide what sample minimum size can be considered as a proper sample size for a language test. Methodology: The data for this quantitative research was 5,250 paper-based TOEFL test scores considered as the population, which includes listening, structure, and reading tests, and it is the most familiar standardized test among EFL researchers. Due to its objective nature, it leaves little chance for bias scores. The score ranged between 30.7% of 417 in the TOEFL scale and 95.7% or 653. Standard error was used as the parameter in deciding the proper sample size. It was the cut-off point when the parameter did not show any obvious change when the sample size was added. We used hierarchical agglomerative clustering with three clusters, determined using 30 indices through the majority rule, in finding out the cut-off point. Main Findings: It was found that the cut-off point is at the sample size of 52 with the range between 46 and 59. Therefore, it can be concluded that the minimum proper sample size for a research study involving a language test is n = 46. Application of this study: The results of this study apply to the area of English language teaching and testing. However, it does not rule out the possibility that the study result applies to tests in other languages. Novelty/Originality of this study: The result of this study should be treated as statistical evidence of the proper sample size to avoid inaccurate or conflicting research results in language teaching where a test is used for analysis

English language · Mathematics education · Population · Sample size determination · Statistical hypothesis testing · Statistical power · Statistics · Test of English as a Foreign Language · Type I and type II errors · Computer Science · Demography · Educational Methods and Analysis · Educational Technology and Assessment · Mathematics · Natural Language Processing Techniques

Tools

Open DOIOpen Access