Supporting online collaborative work at scale: A mixed-methods study of a learning analytics tool

Xavier Ochoa, Vanessa Echeverria, Gladys Carrillo, Vanessa Heredia, Katherine Chiluiza

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

Abstract

Collaborative Learning Analytics (CLA) tools have recently emerged as a potential solution to address the onerous process of monitoring and providing timely feedback on collaboration skills in higher education students. However, prior studies on the efficacy of such tools have mainly been carried out in small, controlled settings. This study aims to measure the impact of a specific CLA tool that can be easily deployed on a larger scale with minimal instructor effort in real-world online group work activities. Additionally, this research examines the potential influence that the characteristics of the collaborative activity may have on the tool's effectiveness. The CLA tool under investigation displays speaking participation time and peer evaluation scores from students engaged in online collaborative activities as part of their regular courses. The tool was evaluated with five instructors and 156 students over the course of one semester. The effects of the tool on students' speaking participation and peer evaluation scores were quantitatively measured and tested. A qualitative analysis of reflections from both students and instructors provided supplementary information on the quantitative results. The main finding of this study indicates that the tool has an overall small positive impact. The effectiveness of the CLA tool is primarily modulated by the synchronous or asynchronous presence of the instructor, as students tend to interact more naturally and feel less scrutinized in the absence of instructor evaluation. Based on the discussion of the findings, this research suggests design insights to enhance future CLA tools at scale for the purpose of supporting the development of online collaboration skills.

Original languageEnglish
Title of host publicationProceedings of the Tenth ACM Conference on Learning @ Scale
EditorsOlga Viberg, Alejandra Martínez-Monés, Philip Guo
Place of PublicationNew York NY USA
PublisherAssociation for Computing Machinery (ACM)
Pages237-247
Number of pages11
ISBN (Electronic)9798400700255
DOIs
Publication statusPublished - 2023
EventAnnual ACM Conference on Learning @ Scale 2023 - Copenhagen, Denmark
Duration: 20 Jul 202322 Jul 2023
Conference number: 10th
https://dl.acm.org/doi/proceedings/10.1145/3573051 (Proceedings)
https://learningatscale.acm.org/las2023/ (Website)

Conference

ConferenceAnnual ACM Conference on Learning @ Scale 2023
Abbreviated titleL@S '23
Country/TerritoryDenmark
CityCopenhagen
Period20/07/2322/07/23
Internet address

Keywords

  • collaboration skills
  • online collaboration
  • participation time

Cite this