Approaches to Item-Level Data with Cross-Classified Structure
An Illustration with Student Evaluation of Teaching
Bibliographic Data
| ID | 19290872 |
|---|---|
| Authors | Sijia Huang (0000-0001-9598-6949, Indiana University, corresponding author) |
| Year | 2024 |
| Volume | 59 |
| Issue | 3 |
| Pages | 523-542 |
| Publication date | 2024-05-03 |
| Peer Reviewed | Yes |
| Open Access | No |
| Type | ARTICLE |
| Venue | Multivariate Behavioral Research (JOURNAL) |
| Journal identifiers | ISSN: 0027-3171 • E-ISSN: 1532-7906 |
| Publisher | Informa UK Limited (PUBLISHER • GB) |
| DOI | 10.1080/00273171.2023.2288589 |
| PMID | 38351542 |
| OpenAlex | W4391817610 |
| Language | EN |
| References cited | 49 |
Student evaluation of teaching (SET) questionnaires are ubiquitously applied in higher education institutions in North America for both formative and summative purposes. Data collected from SET questionnaires are usually item-level data with cross-classified structure, which are characterized by multivariate categorical outcomes (i.e., multiple Likert-type items in the questionnaires) and cross-classified structure (i.e., non-nested students and instructors). Recently, a new approach, namely the cross-classified IRT model, was proposed for appropriately handling SET data. To inform researchers in higher education, in this article, the cross-classified IRT model, along with three existing approaches applied in SET studies, including the cross-classified random effects model (CCREM), the multilevel item response theory (MLIRT) model, and a two-step integrated strategy, was reviewed. The strengths and weaknesses of each of the four approaches were also discussed. Additionally, the new and existing approaches were compared through an empirical data analysis and a preliminary simulation study. This article concluded by providing general suggestions to researchers for analyzing SET data and discussing limitations and future research directions
Categorical variable · Data mining · Data set · Formative assessment · Item response theory · Likert scale · Machine learning · Mathematics education · Multivariate statistics · Psychometrics · Set (abstract data type) · Statistics · Strengths and weaknesses · Summative assessment · Artificial Intelligence · Computer Science · Evaluation of Teaching Practices · Mathematics · Psychology · Psychometric Methodologies and Testing · Student Assessment and Feedback
Generalized Latent Variable Modeling
On the Validity of Student Evaluation of Teaching
A Rasch Model for Partial Credit Scoring
Multilevel mixed linear model analysis using iterative generalized least squares
Students' evaluations of university teaching
Marginal Maximum Likelihood Estimation of Item Parameters
Estimation of Latent Ability Using a Response Pattern of Graded Scores
A Stochastic Approximation Method
Gender biases in student evaluations of teaching
Maximum Likelihood from Incomplete Data Via the EM Algorithm
Fitting Linear Mixed-Effects Models Using lme4
Validity of students’ evaluations of teaching
On the credibility of the judge
Estimation of a Ramsay-Curve Item Response Theory Model by the Metropolis–Hastings Robbins–Monro Algorithm
Student evaluation of instruction in higher education
Establishing the validity and reliability of course evaluation questionnaires
How reliable are students’ evaluations of teaching quality? A variance components approach
The number of feedbacks needed for reliable evaluation. A multilevel analysis of the reliability, stability and generalisability of students’ evaluation of teaching
Student Evaluation of Teaching Effectiveness
Development and validation of a theoretically based, multidimensional questionnaire of student evaluation of university teaching
Psychometric quality of a student evaluation of teaching survey in higher education
The Impacts of Ignoring a Crossed Factor in Analyzing Cross-Classified Data
The Impact of Inappropriate Modeling of Cross-Classified Data Structures
Introduction to Multilevel Item Response Theory Analysis
A rating formulation for ordered response categories
Seeq
| Citation velocity | historical |
|---|---|
| Highly cited | No |