From data to insights: a layered storytelling approach for multimodal learning analytics

Roberto Martinez-Maldonado, Vanessa Echeverria, Gloria Fernandez Nieto, Simon Buckingham Shum

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review


Significant progress to integrate and analyse multimodal data has been carried out in the last years. Yet, little research has tackled the challenge of visualising and supporting the sensemaking of multimodal data to inform teaching and learning. It is naïve to expect that simply by rendering multiple data streams visually, a teacher or learner will be able to make sense of them. This paper introduces an approach to unravel the complexity of multimodal data by organising it into meaningful layers that explain critical insights to teachers and students. The approach is illustrated through the design of two data storytelling prototypes in the context of nursing simulation. Two authentic studies with educators and students identified the potential of the approach to create learning analytics interfaces that communicate insights on team performance, as well as concerns in terms of accountability and automated insights discovery.
Original languageEnglish
Title of host publicationProceedings of the 2020 CHI Conference on Human Factors in Computing Systems
EditorsJoanna McGrenere, Andy Cockburn
Place of PublicationNew York NY USA
PublisherAssociation for Computing Machinery (ACM)
Number of pages15
ISBN (Electronic)9781450367080
Publication statusPublished - 2020
EventInternational Conference on Human Factors in Computing Systems 2020 - Honolulu , United States of America
Duration: 25 Apr 202030 Apr 2020
Conference number: 38th (Website) (Proceedings)


ConferenceInternational Conference on Human Factors in Computing Systems 2020
Abbreviated titleCHI 2020
CountryUnited States of America
Internet address


  • CSCW
  • teamwork
  • visualization
  • data storytelling

Cite this