Examining manual and semi-automated methods of analysing MOOC data for computing education

Michael Morgan; Aletta Nylén; Matthew Butler; Anna Eckerdal; Neena Thota; Päivi Kinnunen

doi:10.1145/3141880.3141887

Examining manual and semi-automated methods of analysing MOOC data for computing education

Michael Morgan, Aletta Nylén, Matthew Butler, Anna Eckerdal, Neena Thota, Päivi Kinnunen

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › Research › peer-review

2 Citations (Scopus)

Abstract

We examine a semi-automated approach to the analysis of data from MOOC discussion forums. Previous research had analysed a sample of discussion forum data and developed a manual analysis framework, however this process can be very time consuming, especially given the class size of some online courses. Therefore it is important to investigate appropriate and automated analysis techniques to improve timeliness of analysis and to reveal the topics that emerge from a semi-automated process. An analysis of a data set from a coding MOOC in 2015 using the automated Structural Topic Modeling (STM) technique in R is described and contrasted against a manual analysis conducted on a segment of data from the same course in 2014. The types of analyses available and the relevance to computing education research is highlighted, with a focus on providing a discussion of the contrasting capabilities of each approach. The aim is to enable computing education researchers to assess the relevance of these techniques for further work.

Original language	English
Title of host publication	Proceedings
Subtitle of host publication	17th Koli Calling Conference on Computing Education Research - Koli Calling 2017
Editors	Calkin Suero Montero, Mike Joy
Place of Publication	New York NY USA
Publisher	Association for Computing Machinery (ACM)
Pages	153-157
Number of pages	5
ISBN (Print)	9781450353014
DOIs	https://doi.org/10.1145/3141880.3141887
Publication status	Published - 16 Nov 2017
Event	Baltic Sea Conference on Computing Education Research 2017 - Koli, Finland Duration: 16 Nov 2017 → 19 Nov 2017 Conference number: 17th http://www.kolicalling.fi/index.php/previous-koli-calling-conferences/koli-calling-2017/general-information-2017

Conference

Conference	Baltic Sea Conference on Computing Education Research 2017
Abbreviated title	Koli Calling 2017
Country/Territory	Finland
City	Koli
Period	16/11/17 → 19/11/17
Internet address	http://www.kolicalling.fi/index.php/previous-koli-calling-conferences/koli-calling-2017/general-information-2017

Keywords

Data analysis.
MOOC
Online discussion
Programming

Access to Document

10.1145/3141880.3141887

Cite this

Morgan, M., Nylén, A., Butler, M., Eckerdal, A., Thota, N., & Kinnunen, P. (2017). Examining manual and semi-automated methods of analysing MOOC data for computing education. In C. S. Montero, & M. Joy (Eds.), Proceedings: 17th Koli Calling Conference on Computing Education Research - Koli Calling 2017 (pp. 153-157). Association for Computing Machinery (ACM). https://doi.org/10.1145/3141880.3141887

@inproceedings{12b4d9993e634c2d939ad5ba3d5757fa,

title = "Examining manual and semi-automated methods of analysing MOOC data for computing education",

abstract = "We examine a semi-automated approach to the analysis of data from MOOC discussion forums. Previous research had analysed a sample of discussion forum data and developed a manual analysis framework, however this process can be very time consuming, especially given the class size of some online courses. Therefore it is important to investigate appropriate and automated analysis techniques to improve timeliness of analysis and to reveal the topics that emerge from a semi-automated process. An analysis of a data set from a coding MOOC in 2015 using the automated Structural Topic Modeling (STM) technique in R is described and contrasted against a manual analysis conducted on a segment of data from the same course in 2014. The types of analyses available and the relevance to computing education research is highlighted, with a focus on providing a discussion of the contrasting capabilities of each approach. The aim is to enable computing education researchers to assess the relevance of these techniques for further work.",

keywords = "Data analysis., MOOC, Online discussion, Programming",

author = "Michael Morgan and Aletta Nyl{\'e}n and Matthew Butler and Anna Eckerdal and Neena Thota and P{\"a}ivi Kinnunen",

year = "2017",

month = nov,

day = "16",

doi = "10.1145/3141880.3141887",

language = "English",

isbn = "9781450353014",

pages = "153--157",

editor = "Montero, {Calkin Suero} and Mike Joy",

booktitle = "Proceedings",

publisher = "Association for Computing Machinery (ACM)",

address = "United States of America",

note = "Baltic Sea Conference on Computing Education Research 2017, Koli Calling 2017 ; Conference date: 16-11-2017 Through 19-11-2017",

url = "http://www.kolicalling.fi/index.php/previous-koli-calling-conferences/koli-calling-2017/general-information-2017",

}

Morgan, M, Nylén, A, Butler, M, Eckerdal, A, Thota, N & Kinnunen, P 2017, Examining manual and semi-automated methods of analysing MOOC data for computing education. in CS Montero & M Joy (eds), Proceedings: 17th Koli Calling Conference on Computing Education Research - Koli Calling 2017. Association for Computing Machinery (ACM), New York NY USA, pp. 153-157, Baltic Sea Conference on Computing Education Research 2017, Koli, Finland, 16/11/17. https://doi.org/10.1145/3141880.3141887

Examining manual and semi-automated methods of analysing MOOC data for computing education. / Morgan, Michael; Nylén, Aletta; Butler, Matthew et al.
Proceedings: 17th Koli Calling Conference on Computing Education Research - Koli Calling 2017. ed. / Calkin Suero Montero; Mike Joy. New York NY USA: Association for Computing Machinery (ACM), 2017. p. 153-157.

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › Research › peer-review

TY - GEN

T1 - Examining manual and semi-automated methods of analysing MOOC data for computing education

AU - Morgan, Michael

AU - Nylén, Aletta

AU - Butler, Matthew

AU - Eckerdal, Anna

AU - Thota, Neena

AU - Kinnunen, Päivi

N1 - Conference code: 17th

PY - 2017/11/16

Y1 - 2017/11/16

N2 - We examine a semi-automated approach to the analysis of data from MOOC discussion forums. Previous research had analysed a sample of discussion forum data and developed a manual analysis framework, however this process can be very time consuming, especially given the class size of some online courses. Therefore it is important to investigate appropriate and automated analysis techniques to improve timeliness of analysis and to reveal the topics that emerge from a semi-automated process. An analysis of a data set from a coding MOOC in 2015 using the automated Structural Topic Modeling (STM) technique in R is described and contrasted against a manual analysis conducted on a segment of data from the same course in 2014. The types of analyses available and the relevance to computing education research is highlighted, with a focus on providing a discussion of the contrasting capabilities of each approach. The aim is to enable computing education researchers to assess the relevance of these techniques for further work.

AB - We examine a semi-automated approach to the analysis of data from MOOC discussion forums. Previous research had analysed a sample of discussion forum data and developed a manual analysis framework, however this process can be very time consuming, especially given the class size of some online courses. Therefore it is important to investigate appropriate and automated analysis techniques to improve timeliness of analysis and to reveal the topics that emerge from a semi-automated process. An analysis of a data set from a coding MOOC in 2015 using the automated Structural Topic Modeling (STM) technique in R is described and contrasted against a manual analysis conducted on a segment of data from the same course in 2014. The types of analyses available and the relevance to computing education research is highlighted, with a focus on providing a discussion of the contrasting capabilities of each approach. The aim is to enable computing education researchers to assess the relevance of these techniques for further work.

KW - Data analysis.

KW - MOOC

KW - Online discussion

KW - Programming

UR - http://www.scopus.com/inward/record.url?scp=85040698000&partnerID=8YFLogxK

U2 - 10.1145/3141880.3141887

DO - 10.1145/3141880.3141887

M3 - Conference Paper

AN - SCOPUS:85040698000

SN - 9781450353014

SP - 153

EP - 157

BT - Proceedings

A2 - Montero, Calkin Suero

A2 - Joy, Mike

PB - Association for Computing Machinery (ACM)

CY - New York NY USA

T2 - Baltic Sea Conference on Computing Education Research 2017

Y2 - 16 November 2017 through 19 November 2017

ER -

Morgan M, Nylén A, Butler M, Eckerdal A, Thota N, Kinnunen P. Examining manual and semi-automated methods of analysing MOOC data for computing education. In Montero CS, Joy M, editors, Proceedings: 17th Koli Calling Conference on Computing Education Research - Koli Calling 2017. New York NY USA: Association for Computing Machinery (ACM). 2017. p. 153-157 doi: 10.1145/3141880.3141887

Examining manual and semi-automated methods of analysing MOOC data for computing education

Abstract

Conference

Keywords

Access to Document

Other files and links

Cite this