Automated analysis of cognitive presence in online discussions written in Portuguese

Valter Neto, Vitor Rolim, Rafael Ferreira, Vitomir Kovanović, Dragan Gašević, Rafael Dueire Lins, Rodrigo Lins

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

2 Citations (Scopus)

Abstract

This paper presents a method for automated content analysis of students’ messages in asynchronous discussions written in Portuguese. In particular, the paper looks at the problem of coding discussion transcripts for the levels of cognitive presence, a key construct in a widely used Community of Inquiry model of online learning. Although there are techniques to coding for cognitive presence in the English language, the literature is still poor in methods for others languages, such as Portuguese. The proposed method uses a set of 87 different features to create a random forest classifier to automatically extract the cognitive phases. The model developed reached Cohen’s κ of.72, which represents a “substantial” agreement, and it is above the Cohen’s κ threshold of.70, commonly used in the literature for determining a reliable quantitative content analysis. This paper also provides some theoretical insights into the nature of cognitive presence by looking at the classification features that were most relevant for distinguishing between the different phases of cognitive presence.

Original languageEnglish
Title of host publicationLifelong Technology-Enhanced Learning
Subtitle of host publication13th European Conference on Technology Enhanced Learning, EC-TEL 2018, Leeds, UK, September 3–5, 2018 Proceedings
EditorsViktoria Pammer-Schindler, Mar Perez-Sanagustin, Hendrik Drachsler, Raymond Elferink, Maren Scheffel
Place of PublicationCham Switzerland
PublisherSpringer
Pages245-261
Number of pages17
Edition1st
ISBN (Electronic)9783319985725
ISBN (Print)9783319985718
DOIs
Publication statusPublished - 2018
EventEuropean Conference on Technology Enhanced Learning 2018 - Leeds, United Kingdom
Duration: 3 Sep 20185 Sep 2018
Conference number: 13th

Publication series

NameLecture Notes in Computer Science
PublisherSpringer
Volume11082
ISSN (Print)0302-9743
ISSN (Electronic)1611-3349

Conference

ConferenceEuropean Conference on Technology Enhanced Learning 2018
Abbreviated titleEC-TEL 2018
CountryUnited Kingdom
CityLeeds
Period3/09/185/09/18

Keywords

  • Community of Inquiry (CoI) model
  • Content analytics
  • Online discussions
  • Text classification

Cite this

Neto, V., Rolim, V., Ferreira, R., Kovanović, V., Gašević, D., Dueire Lins, R., & Lins, R. (2018). Automated analysis of cognitive presence in online discussions written in Portuguese. In V. Pammer-Schindler, M. Perez-Sanagustin, H. Drachsler, R. Elferink, & M. Scheffel (Eds.), Lifelong Technology-Enhanced Learning: 13th European Conference on Technology Enhanced Learning, EC-TEL 2018, Leeds, UK, September 3–5, 2018 Proceedings (1st ed., pp. 245-261). (Lecture Notes in Computer Science; Vol. 11082). Cham Switzerland: Springer. https://doi.org/10.1007/978-3-319-98572-5_19
Neto, Valter ; Rolim, Vitor ; Ferreira, Rafael ; Kovanović, Vitomir ; Gašević, Dragan ; Dueire Lins, Rafael ; Lins, Rodrigo. / Automated analysis of cognitive presence in online discussions written in Portuguese. Lifelong Technology-Enhanced Learning: 13th European Conference on Technology Enhanced Learning, EC-TEL 2018, Leeds, UK, September 3–5, 2018 Proceedings. editor / Viktoria Pammer-Schindler ; Mar Perez-Sanagustin ; Hendrik Drachsler ; Raymond Elferink ; Maren Scheffel. 1st. ed. Cham Switzerland : Springer, 2018. pp. 245-261 (Lecture Notes in Computer Science).
@inproceedings{278f9060ff034761be8a094074f5eac3,
title = "Automated analysis of cognitive presence in online discussions written in Portuguese",
abstract = "This paper presents a method for automated content analysis of students’ messages in asynchronous discussions written in Portuguese. In particular, the paper looks at the problem of coding discussion transcripts for the levels of cognitive presence, a key construct in a widely used Community of Inquiry model of online learning. Although there are techniques to coding for cognitive presence in the English language, the literature is still poor in methods for others languages, such as Portuguese. The proposed method uses a set of 87 different features to create a random forest classifier to automatically extract the cognitive phases. The model developed reached Cohen’s κ of.72, which represents a “substantial” agreement, and it is above the Cohen’s κ threshold of.70, commonly used in the literature for determining a reliable quantitative content analysis. This paper also provides some theoretical insights into the nature of cognitive presence by looking at the classification features that were most relevant for distinguishing between the different phases of cognitive presence.",
keywords = "Community of Inquiry (CoI) model, Content analytics, Online discussions, Text classification",
author = "Valter Neto and Vitor Rolim and Rafael Ferreira and Vitomir Kovanović and Dragan Gašević and {Dueire Lins}, Rafael and Rodrigo Lins",
year = "2018",
doi = "10.1007/978-3-319-98572-5_19",
language = "English",
isbn = "9783319985718",
series = "Lecture Notes in Computer Science",
publisher = "Springer",
pages = "245--261",
editor = "Viktoria Pammer-Schindler and Mar Perez-Sanagustin and Hendrik Drachsler and Raymond Elferink and Maren Scheffel",
booktitle = "Lifelong Technology-Enhanced Learning",
edition = "1st",

}

Neto, V, Rolim, V, Ferreira, R, Kovanović, V, Gašević, D, Dueire Lins, R & Lins, R 2018, Automated analysis of cognitive presence in online discussions written in Portuguese. in V Pammer-Schindler, M Perez-Sanagustin, H Drachsler, R Elferink & M Scheffel (eds), Lifelong Technology-Enhanced Learning: 13th European Conference on Technology Enhanced Learning, EC-TEL 2018, Leeds, UK, September 3–5, 2018 Proceedings. 1st edn, Lecture Notes in Computer Science, vol. 11082, Springer, Cham Switzerland, pp. 245-261, European Conference on Technology Enhanced Learning 2018, Leeds, United Kingdom, 3/09/18. https://doi.org/10.1007/978-3-319-98572-5_19

Automated analysis of cognitive presence in online discussions written in Portuguese. / Neto, Valter; Rolim, Vitor; Ferreira, Rafael; Kovanović, Vitomir; Gašević, Dragan; Dueire Lins, Rafael; Lins, Rodrigo.

Lifelong Technology-Enhanced Learning: 13th European Conference on Technology Enhanced Learning, EC-TEL 2018, Leeds, UK, September 3–5, 2018 Proceedings. ed. / Viktoria Pammer-Schindler; Mar Perez-Sanagustin; Hendrik Drachsler; Raymond Elferink; Maren Scheffel. 1st. ed. Cham Switzerland : Springer, 2018. p. 245-261 (Lecture Notes in Computer Science; Vol. 11082).

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

TY - GEN

T1 - Automated analysis of cognitive presence in online discussions written in Portuguese

AU - Neto, Valter

AU - Rolim, Vitor

AU - Ferreira, Rafael

AU - Kovanović, Vitomir

AU - Gašević, Dragan

AU - Dueire Lins, Rafael

AU - Lins, Rodrigo

PY - 2018

Y1 - 2018

N2 - This paper presents a method for automated content analysis of students’ messages in asynchronous discussions written in Portuguese. In particular, the paper looks at the problem of coding discussion transcripts for the levels of cognitive presence, a key construct in a widely used Community of Inquiry model of online learning. Although there are techniques to coding for cognitive presence in the English language, the literature is still poor in methods for others languages, such as Portuguese. The proposed method uses a set of 87 different features to create a random forest classifier to automatically extract the cognitive phases. The model developed reached Cohen’s κ of.72, which represents a “substantial” agreement, and it is above the Cohen’s κ threshold of.70, commonly used in the literature for determining a reliable quantitative content analysis. This paper also provides some theoretical insights into the nature of cognitive presence by looking at the classification features that were most relevant for distinguishing between the different phases of cognitive presence.

AB - This paper presents a method for automated content analysis of students’ messages in asynchronous discussions written in Portuguese. In particular, the paper looks at the problem of coding discussion transcripts for the levels of cognitive presence, a key construct in a widely used Community of Inquiry model of online learning. Although there are techniques to coding for cognitive presence in the English language, the literature is still poor in methods for others languages, such as Portuguese. The proposed method uses a set of 87 different features to create a random forest classifier to automatically extract the cognitive phases. The model developed reached Cohen’s κ of.72, which represents a “substantial” agreement, and it is above the Cohen’s κ threshold of.70, commonly used in the literature for determining a reliable quantitative content analysis. This paper also provides some theoretical insights into the nature of cognitive presence by looking at the classification features that were most relevant for distinguishing between the different phases of cognitive presence.

KW - Community of Inquiry (CoI) model

KW - Content analytics

KW - Online discussions

KW - Text classification

UR - http://www.scopus.com/inward/record.url?scp=85053215051&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-98572-5_19

DO - 10.1007/978-3-319-98572-5_19

M3 - Conference Paper

SN - 9783319985718

T3 - Lecture Notes in Computer Science

SP - 245

EP - 261

BT - Lifelong Technology-Enhanced Learning

A2 - Pammer-Schindler, Viktoria

A2 - Perez-Sanagustin, Mar

A2 - Drachsler, Hendrik

A2 - Elferink, Raymond

A2 - Scheffel, Maren

PB - Springer

CY - Cham Switzerland

ER -

Neto V, Rolim V, Ferreira R, Kovanović V, Gašević D, Dueire Lins R et al. Automated analysis of cognitive presence in online discussions written in Portuguese. In Pammer-Schindler V, Perez-Sanagustin M, Drachsler H, Elferink R, Scheffel M, editors, Lifelong Technology-Enhanced Learning: 13th European Conference on Technology Enhanced Learning, EC-TEL 2018, Leeds, UK, September 3–5, 2018 Proceedings. 1st ed. Cham Switzerland: Springer. 2018. p. 245-261. (Lecture Notes in Computer Science). https://doi.org/10.1007/978-3-319-98572-5_19