Automatically Measuring Question Authenticity in Real-World Classrooms

Bibliographic Data

ID9735533
AuthorsSean Kelly (0000-0002-2415-0375, University of Pittsburgh, PA, corresponding author), Andrew M Olney (0000-0003-4204-6667, University of Memphis, TN), Patrick Donnelly (California State University–Chico, CA), Patrick J Donnelly (0000-0003-1033-5931, California State University, Chico), Martin Nystrand (University of Wisconsin–Madison, WI), Sidney K D’Mello (0000-0003-0347-2807, University of Colorado, Boulder, CO)
Year2018
Volume47
Issue7
Pages451-464
Publication date2018-10-01
Peer ReviewedYes
Open AccessYes
TypeARTICLE
VenueEducational Researcher (JOURNAL)
Journal identifiersISSN: 0013-189X • E-ISSN: 1935-102X
PublisherAmerican Educational Research Association (AERA) (PUBLISHER)
DOI10.3102/0013189x18785613
OpenAlexW2809787282
LanguageEN
Citations received29
References cited45

Analyzing the quality of classroom talk is central to educational research and improvement efforts. In particular, the presence of authentic teacher questions, where answers are not predetermined by the teacher, helps constitute and serves as a marker of productive classroom discourse. Further, authentic questions can be cultivated to improve teaching effectiveness and consequently student achievement. Unfortunately, current methods to measure question authenticity do not scale because they rely on human observations or coding of teacher discourse. To address this challenge, we set out to use automatic speech recognition, natural language processing, and machine learning to train computers to detect authentic questions in real-world classrooms automatically. Our methods were iteratively refined using classroom audio and human-coded observational data from two sources: (a) a large archival database of text transcripts of 451 observations from 112 classrooms; and (b) a newly collected sample of 132 high-quality audio recordings from 27 classrooms, obtained under technical constraints that anticipate large-scale automated data collection and analysis. Correlations between human-coded and computer-coded authenticity at the classroom level were sufficiently high ( r = .602 for archival transcripts and .687 for audio recordings) to provide a valuable complement to human coding in research efforts

Audio equipment · Coding (social sciences · Complement (music · Data collection · Mathematics education · Observational study · Quality (philosophy · Scale (ratio · Set (abstract data type · Sociology · Computer Science · Innovative Teaching and Learning Methods · Psychology · Speech and dialogue systems · Topic Modeling

Tools

Open DOISci-Hub