How good is my feedback?: a content analysis of written feedback

Anderson Pinheiro Cavalcanti; Arthur Diego; Rafael Ferreira Mello; Katerina Mangaroska; André Nascimento; Fred Freitas; Dragan Gaševic

doi:10.1145/3375462.3375477

How good is my feedback? a content analysis of written feedback

Anderson Pinheiro Cavalcanti, Arthur Diego, Rafael Ferreira Mello, Katerina Mangaroska, André Nascimento, Fred Freitas, Dragan Gaševic

Department of Data Science & AI

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › Research › peer-review

14 Citations (Scopus)

Abstract

Feedback is a crucial element in helping students identify gaps and assess their learning progress. In online courses, feedback becomes even more critical as it is one of the resources where the teacher interacts directly with the student. However, with the growing number of students enrolled in online learning, it becomes a challenge for instructors to provide good quality feedback that helps the student self-regulate. In this context, this paper proposed a content analysis of feedback text provided by instructors based on different indicators of good feedback. A random forest classifier was trained and evaluated at different feedback levels. The results achieved outcomes up to 87% and 0.39 of accuracy and Cohen's κ, respectively. The paper also provides insights into the most influential textual features of feedback that predict feedback quality.

Original language	English
Title of host publication	LAK 2020 Conference Proceedings
Editors	Vitomir Kovanović, Maren Scheffel, Niels Pinkwart, Katrien Verbert
Place of Publication	New York NY USA
Publisher	Association for Computing Machinery (ACM)
Pages	428-437
Number of pages	10
ISBN (Electronic)	9781450377126
DOIs	https://doi.org/10.1145/3375462.3375477
Publication status	Published - 2020
Event	International Conference on Learning Analytics and Knowledge 2020 - Frankfurt, Germany Duration: 23 Mar 2020 → 27 Mar 2020 Conference number: 10th https://lak20.solaresearch.org (Website) https://dl-acm-org.ezproxy.lib.monash.edu.au/doi/proceedings/10.1145/3375462 (Website)

Conference

Conference	International Conference on Learning Analytics and Knowledge 2020
Abbreviated title	LAK 2020
Country/Territory	Germany
City	Frankfurt
Period	23/03/20 → 27/03/20
Internet address	https://lak20.solaresearch.org (Website) https://dl-acm-org.ezproxy.lib.monash.edu.au/doi/proceedings/10.1145/3375462 (Website)

Keywords

Content Analysis
Feedback
Learning Analytics
Online learning

Access to Document

10.1145/3375462.3375477

Cite this

Cavalcanti, A. P., Diego, A., Mello, R. F., Mangaroska, K., Nascimento, A., Freitas, F., & Gaševic, D. (2020). How good is my feedback? a content analysis of written feedback. In V. Kovanović, M. Scheffel, N. Pinkwart, & K. Verbert (Eds.), LAK 2020 Conference Proceedings (pp. 428-437). Association for Computing Machinery (ACM). https://doi.org/10.1145/3375462.3375477

@inproceedings{51d55b482f354d17ba270d60a6080cd0,

title = "How good is my feedback?: a content analysis of written feedback",

abstract = "Feedback is a crucial element in helping students identify gaps and assess their learning progress. In online courses, feedback becomes even more critical as it is one of the resources where the teacher interacts directly with the student. However, with the growing number of students enrolled in online learning, it becomes a challenge for instructors to provide good quality feedback that helps the student self-regulate. In this context, this paper proposed a content analysis of feedback text provided by instructors based on different indicators of good feedback. A random forest classifier was trained and evaluated at different feedback levels. The results achieved outcomes up to 87% and 0.39 of accuracy and Cohen's κ, respectively. The paper also provides insights into the most influential textual features of feedback that predict feedback quality.",

keywords = "Content Analysis, Feedback, Learning Analytics, Online learning",

author = "Cavalcanti, {Anderson Pinheiro} and Arthur Diego and Mello, {Rafael Ferreira} and Katerina Mangaroska and Andr{\'e} Nascimento and Fred Freitas and Dragan Ga{\v s}evic",

year = "2020",

doi = "10.1145/3375462.3375477",

language = "English",

pages = "428--437",

editor = "Kovanovi{\'c}, {Vitomir } and Scheffel, {Maren } and Pinkwart, {Niels } and Verbert, {Katrien }",

booktitle = "LAK 2020 Conference Proceedings",

publisher = "Association for Computing Machinery (ACM)",

address = "United States of America",

note = "International Conference on Learning Analytics and Knowledge 2020, LAK 2020 ; Conference date: 23-03-2020 Through 27-03-2020",

url = "https://lak20.solaresearch.org, https://dl-acm-org.ezproxy.lib.monash.edu.au/doi/proceedings/10.1145/3375462",

}

Cavalcanti, AP, Diego, A, Mello, RF, Mangaroska, K, Nascimento, A, Freitas, F & Gaševic, D 2020, How good is my feedback? a content analysis of written feedback. in V Kovanović, M Scheffel, N Pinkwart & K Verbert (eds), LAK 2020 Conference Proceedings . Association for Computing Machinery (ACM), New York NY USA, pp. 428-437, International Conference on Learning Analytics and Knowledge 2020, Frankfurt, Hesse, Germany, 23/03/20. https://doi.org/10.1145/3375462.3375477

How good is my feedback? a content analysis of written feedback. / Cavalcanti, Anderson Pinheiro; Diego, Arthur; Mello, Rafael Ferreira et al.

LAK 2020 Conference Proceedings . ed. / Vitomir Kovanović; Maren Scheffel; Niels Pinkwart; Katrien Verbert. New York NY USA : Association for Computing Machinery (ACM), 2020. p. 428-437.

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › Research › peer-review

TY - GEN

T1 - How good is my feedback?

T2 - International Conference on Learning Analytics and Knowledge 2020

AU - Cavalcanti, Anderson Pinheiro

AU - Diego, Arthur

AU - Mello, Rafael Ferreira

AU - Mangaroska, Katerina

AU - Nascimento, André

AU - Freitas, Fred

AU - Gaševic, Dragan

N1 - Conference code: 10th

PY - 2020

Y1 - 2020

N2 - Feedback is a crucial element in helping students identify gaps and assess their learning progress. In online courses, feedback becomes even more critical as it is one of the resources where the teacher interacts directly with the student. However, with the growing number of students enrolled in online learning, it becomes a challenge for instructors to provide good quality feedback that helps the student self-regulate. In this context, this paper proposed a content analysis of feedback text provided by instructors based on different indicators of good feedback. A random forest classifier was trained and evaluated at different feedback levels. The results achieved outcomes up to 87% and 0.39 of accuracy and Cohen's κ, respectively. The paper also provides insights into the most influential textual features of feedback that predict feedback quality.

AB - Feedback is a crucial element in helping students identify gaps and assess their learning progress. In online courses, feedback becomes even more critical as it is one of the resources where the teacher interacts directly with the student. However, with the growing number of students enrolled in online learning, it becomes a challenge for instructors to provide good quality feedback that helps the student self-regulate. In this context, this paper proposed a content analysis of feedback text provided by instructors based on different indicators of good feedback. A random forest classifier was trained and evaluated at different feedback levels. The results achieved outcomes up to 87% and 0.39 of accuracy and Cohen's κ, respectively. The paper also provides insights into the most influential textual features of feedback that predict feedback quality.

KW - Content Analysis

KW - Feedback

KW - Learning Analytics

KW - Online learning

UR - http://www.scopus.com/inward/record.url?scp=85082396357&partnerID=8YFLogxK

U2 - 10.1145/3375462.3375477

DO - 10.1145/3375462.3375477

M3 - Conference Paper

AN - SCOPUS:85082396357

SP - 428

EP - 437

BT - LAK 2020 Conference Proceedings

A2 - Kovanović, Vitomir

A2 - Scheffel, Maren

A2 - Pinkwart, Niels

A2 - Verbert, Katrien

PB - Association for Computing Machinery (ACM)

CY - New York NY USA

Y2 - 23 March 2020 through 27 March 2020