Distributional properties of the statistic of online student evaluations the mean does not mean what you think it means
Bibliographic Data
| ID | 14690441 |
|---|---|
| Authors | Brady O Smith (0000-0003-1286-0852, University of Nebraska at Omaha, Omaha, NE, USA, corresponding author), Dustin R White (0000-0002-2566-5338, University of Nebraska at Omaha, Omaha, NE, USA), Jamie Wagner (0000-0003-0527-8282, University of Nebraska at Omaha, Omaha, NE, USA), Patricia Kuzyk (Washington State University, Pullman, WA, USA), Alex Prera (Washington State University, Pullman, WA, USA) |
| Year | 2023 |
| Volume | 48 |
| Issue | 11 |
| Pages | 1678-1695 |
| Publication date | 2023-11-02 |
| Peer Reviewed | Yes |
| Open Access | No |
| Type | ARTICLE |
| Venue | Studies in Higher Education (JOURNAL) |
| Journal identifiers | ISSN: 0307-5079 • E-ISSN: 1470-174X |
| Publisher | Informa UK Limited (PUBLISHER • GB) |
| DOI | 10.1080/03075079.2023.2211079 |
| OpenAlex | W4376108812 |
| Language | EN |
| References cited | 23 |
Student Evaluations of Teaching (SETs) are an integral part of evaluating course outcomes. They are routinely used to evaluate teaching quality for the purposes of reappointment, promotion, and tenure (RPT), annual review, and the rehiring of adjunct faculty and lecturers. These evaluations are often based almost entirely on the mean or proportion of the ordinal overall score with no regard to statistical noise. This study examines the distribution of the statistic (mean or proportion) when SETs are administered online and in-person. Using non-parametric procedures, we show that the size of the 95% confidence interval of the statistic is a function of response rates. Prior to COVID-19, online administration of SETs resulted in significantly more uncertainty than in-person administration because the in-person response rates were higher. Due to a decrease in in-person response rates in the post-COVID vaccine period, both methods result in significant levels of uncertainty of the true statistic value. In classes of fewer than 30 students, the 95% confidence interval of the statistic is wide enough for instructors to be considered for a teaching award in one semester or below average in another semester, while holding teaching quality constant
Confidence interval · Econometrics · Economics · Higher education · Mathematics education · Statistic · Statistics · Evaluation of Teaching Practices · Innovations in Educational Methods · Mathematics · Online and Blended Learning · Psychology
An Introduction to the Bootstrap
Student Evaluations of Teaching: Are They Related to What Students Learn?
Locally Weighted Regression
What’s in a Name
Gender and cultural bias in student evaluations
Gender biases in student evaluations of teaching
Student evaluations of college professors
Attitudes of Business Faculty Towards Two Methods of Collecting Teaching Evaluations
Online versus traditional teaching evaluation
Gathering faculty teaching evaluations by in‐class and online surveys
Can we trust teaching evaluations when response rates are not high? Implications from a Monte Carlo simulation
Students' Perceptions of Expressiveness
Does Professor Quality Matter? Evidence from Random Assignment of Students to Professors
| Citation velocity | historical |
|---|---|
| Highly cited | No |