Comparing reusable, atomic feedback with classic feedback on a linear equations task using text mining and qualitative techniques
Bibliographic Data
| ID | 21297585 |
|---|---|
| Authors | Filip Moons (0000-0002-5368-3429, Freudenthal Institute Utrecht University Utrecht The Netherlands, corresponding author), Alexander Holvoet (0000-0002-7549-3247, Faculty of Economics and Business KU Leuven Leuven Belgium), Katrin Klingbeil (0009-0003-6552-0701, Didaktik der Mathematik (Fakultät für Mathematik) Universität Duisburg‐Essen Essen Germany), Ellen Vandervieren (0000-0003-1569-5274, Antwerp School of Education University of Antwerp Antwerp Belgium) |
| Year | 2024 |
| Volume | 55 |
| Issue | 5 |
| Pages | 2257-2277 |
| Publication date | 2024-09-01 |
| Open Access | Yes |
| Type | ARTICLE |
| Venue | British Journal of Educational Technology (JOURNAL) |
| Journal identifiers | ISSN: 0007-1013 • E-ISSN: 1467-8535 |
| Publisher | Wiley (PUBLISHER • GB) |
| DOI | 10.1111/bjet.13447 |
| OpenAlex | W4392157332 |
| Language | EN |
| Citations received | 1 |
| References cited | 36 |
In this crossover experiment, we investigated the impact of a statement bank, enabling the reuse of previously written feedback (SA condition), on 45 math teachers' feedback for 60 completed linear equation tests, compared to traditional pen‐and‐paper feedback (PP condition). In the SA condition, teachers were encouraged to use atomic feedback, a set of formulation requirements that makes feedback items significantly more reusable. A previous study found that significantly more feedback was written in the SA condition but did not investigate the content of the feedback. To address this gap, we employed a novel approach of combining text mining with qualitative methods. Results indicate similar wording and sentiments in both conditions. However, SA feedback was more elaborate yet general, focusing on major and minor strengths and deficits, while PP feedback was shorter but more concrete, emphasising main issues. Despite low feedback quality in both conditions, the statement bank led to less effective diagnostic activities, implying that teachers' careless use of statement banks, although convenient, might lead to lower‐quality feedback. Practitioner notes What is already known about this topic High‐quality feedback should strike a balance between the volume and focus on the main issues, as more feedback does not necessarily equate to better feedback. Feedback should analyse a student's solution whenever possible: interpreting mistakes and communicating that interpretation as feedback. Text mining identifies meaningful patterns and new insights in text using computer algorithms. When teachers can reuse already given feedback using a software tool (statement bank), they tend to write more feedback instead of saving time. What this paper adds Feedback is compared when teachers could use a tool to reuse already given feedback (referred to as ‘statement banks’) versus a scenario without such a tool. Both approaches observed similar word frequencies, sentiments and amounts of erroneous, descriptive and corrective feedback. However, feedback with a statement bank tended to be more elaborate yet less specific to individual student solutions. In contrast, feedback without the tool was shorter but more concrete, focusing on main issues. Overall, the tool for reusing feedback directed teachers towards less effective diagnostic activities. The paper introduces a novel methodological approach by combining text mining with qualitative techniques in educational research. While text mining provides an overall understanding of differences and similarities in feedback approaches, qualitative methods are essential for in‐depth analysis of content characteristics and feedback quality. Implications for practice and/or policy Statement banks can support teachers by giving more feedback, but in order to improve feedback quality, further measures are necessary (eg, improving pedagogical content knowledge). Teachers may not confuse handiness with quality: statement banks can help, but when used carelessly, teachers tend to describe and correct students' work instead of analysing underlying (mis‐)conceptions using it. Continued attention to feedback quality remains necessary when using such tools
Human–computer interaction · Interpretation (philosophy) · Linguistics · Management science · Mathematics education · Peer feedback · Problem statement · Programming language · Quality (philosophy) · Reuse · Set (abstract data type) · Statement (logic) · Task (project management) · Computer Science · Educational Technology and Assessment · Psychology · Software · Software Engineering Research · Student Assessment and Feedback
Supporting Learners' Agentic Engagement With Feedback
How sample size influences research outcomes
Making Sense of Assessment Feedback in Higher Education
Pedagogical content knowledge
Chapter 9 Classification or Wonder? Coding as an Analytic Practice in Qualitative Research
Focus on Formative Feedback
Improving the quality of teaching by utilising written student feedback
Towards the successful game-based learning
Effects of timing of formative feedback in computer‐assisted learning environments
Text Mining in Practice with R
Feedback
Beyond feedback
Response of students to statement bank feedback
Using statement banks to return online feedback
Compatibility between Text Mining and Qualitative Research in the Perspectives of Grounded Theory, Content Analysis, and Reliability
The Power of Feedback
| Citations per year | 1 |
|---|---|
| Citation span | 2025 - 2025 (1) |
| Highly cited | No |