Evidence‐based multimodal learning analytics for feedback and reflection in collaborative learning
Bibliographic Data
Multimodal learning analytics (MMLA) offers the potential to provide evidence‐based insights into complex learning phenomena such as collaborative learning. Yet, few MMLA applications have closed the learning analytics loop by being evaluated in real‐world educational settings. This study evaluates the effectiveness of an MMLA solution in enhancing feedback and reflection within a complex and highly dynamic collaborative learning environment. A two‐year longitudinal study was conducted with 399 students and 17 teachers, utilising an MMLA system in reflective debriefings in the context of healthcare education. We analysed the survey data of 74 students and 11 teachers regarding their perceptions of the MMLA system. We applied the Evaluation Framework for Learning Analytics, augmented by complexity, accuracy and trust measures, to assess both teachers' and students' perspectives. The findings illustrated that teachers and students both had generally positive perceptions of the MMLA solution. Teachers found the MMLA solution helpful in facilitating feedback provision and reflection during debriefing sessions. Similarly, students found the MMLA solution effective in providing clarity on the data collected, stimulating reflection on their learning behaviours, and prompting considerations for adaptation in their learning behaviours. However, the complexity of the MMLA solution and the need for qualitative measures of communication emerged as areas for improvement. Additionally, the study highlighted the importance of data accuracy, transparency, and privacy protection to maintain user trust. The findings provide valuable contributions to advancing our understanding of the use of MMLA in supporting feedback and reflection practices in intricate collaborative learning while identifying avenues for further research and improvement. We also provided several insights and practical recommendations for successful MMLA implementation in authentic learning contexts. Practitioner notes What is currently known about this topic Multimodal learning analytics (MMLA) seeks to generate data‐informed insights about learners' metacognitive and emotional states as well as their learning behaviours, by utilising intricate physical and physiological signals. MMLA has not only pioneered novel data analytic methods but also aspired to complete the learning analytics loop by crafting innovative, tangible solutions that relay these insights to the concerned stakeholders. A prominent direction within MMLA research has been the formulation of tools to support feedback and reflection in collaborative learning scenarios, given MMLA's capacity to discern intricate and dynamic learning behaviours. What this paper adds Teachers' and students' positive perceptions of an MMLA implementation in stimulating considerations of adaptations in their pedagogical practices and learning behaviours, respectively. Empirical evidence supporting the potential of MMLA in assisting teachers to facilitate students' reflective practices during intricate collaborative learning scenarios. The importance of addressing issues related to design complexity, interpretability for users with disabilities, aggregated data representation, and concerns related to trust for building a practical MMLA solution in real learning settings. Implications for practice and/or policy The MMLA solution can provide teachers with a comprehensive view of student performance, illuminate areas for improvement, and confirm learning scenario outcomes. The MMLA solution can stimulate students' reflections on their learning behaviours and promote considerations of adaptation in their learning behaviours. Providing clear explanations and guidance on how to interpret analytics, as well as addressing concerns related to data completeness and representation, are essential to maximising utility
Adaptation (eye) · Analytics · CLARITY · Collaborative learning · Context (archaeology) · Data science · Debriefing · Educational technology · Knowledge management · Learning analytics · Mathematics education · Reflection (computer programming) · Transparency (behavior) · Computer Science · E-Learning and Knowledge Management · Online and Blended Learning · Online Learning and Analytics · Psychology · Social Psychology
Physiological synchrony amongst medical residents during crisis management simulation training and a video-based assessment of leaders’ performance
Applying multimodal learning analytics to naturalistic recordings of clinical simulations
Enhancing personalized learning with Artificial Intelligence and analytics
The effects of generative AI agents and scaffolding on enhancing students’ comprehension of visual learning analytics
University Students' Perceptions of a Multimodal AI System for Real‐World Collaboration Analytics
Artificial intelligence and feedback in university education
A bibliometric analysis of interaction in digital learning over sixteen years
Applications of learning analytics in the study of academic performance in higher education
Investigating the impact of ChatGPT ‐assisted feedback on the dynamics and outcomes of online inquiry‐based discussion
Evidence‐based learning analytics
Using epistemic network analysis to examine learners' cognitive processes and emotions in Mooc discussion forums
Developing the Integrated Analysis Matrix (I-AM)
Learning analytics should not promote one size fits all
The current landscape of learning analytics in higher education
Using learning analytics to understand student perceptions of peer feedback
Deploying multimodal learning analytics models to explore the impact of digital distraction and peer learning on student performance
Revealing the hidden structure of physiological states during metacognitive monitoring in collaborative learning
The promise and challenges of multimodal learning analytics
Applications of learning analytics in Latin America
Multimodal learning analytics to investigate cognitive load during online problem solving
Controlled evaluation of a multimodal system to improve oral presentation skills in a real learning setting
The role of indoor positioning analytics in assessment of simulation‐based learning
Multimodal Learning Analytics research with young children
What does physiological synchrony reveal about metacognitive experiences and group performance
Multimodal data capabilities for learning
Estimating the Reliability of a Single-Item Measure
Let’s not forget
One size fits all? What counts as quality practice in (reflexive) thematic analysis
| Unique citing works | 12 |
|---|---|
| Citations per year | 6 |
| Citation span | 2024 - 2026 (3) |
| Citation velocity | current |
| Highly cited | No |
| Citation types | Neutral: 12 |