Examining manual and semi-automated methods of analysing MOOC data for computing education

Michael Morgan, Aletta Nylén, Matthew Butler, Anna Eckerdal, Neena Thota, Päivi Kinnunen

    Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

    Abstract

    We examine a semi-automated approach to the analysis of data from MOOC discussion forums. Previous research had analysed a sample of discussion forum data and developed a manual analysis framework, however this process can be very time consuming, especially given the class size of some online courses. Therefore it is important to investigate appropriate and automated analysis techniques to improve timeliness of analysis and to reveal the topics that emerge from a semi-automated process. An analysis of a data set from a coding MOOC in 2015 using the automated Structural Topic Modeling (STM) technique in R is described and contrasted against a manual analysis conducted on a segment of data from the same course in 2014. The types of analyses available and the relevance to computing education research is highlighted, with a focus on providing a discussion of the contrasting capabilities of each approach. The aim is to enable computing education researchers to assess the relevance of these techniques for further work.

    Original languageEnglish
    Title of host publicationProceedings
    Subtitle of host publication17th Koli Calling Conference on Computing Education Research - Koli Calling 2017
    EditorsCalkin Suero Montero, Mike Joy
    Place of PublicationNew York NY USA
    PublisherAssociation for Computing Machinery (ACM)
    Pages153-157
    Number of pages5
    ISBN (Print)9781450353014
    DOIs
    Publication statusPublished - 16 Nov 2017
    EventBaltic Sea Conference on Computing Education Research 2017 - Koli, Finland
    Duration: 16 Nov 201719 Nov 2017
    Conference number: 17th
    http://www.kolicalling.fi/index.php/previous-koli-calling-conferences/koli-calling-2017/general-information-2017

    Conference

    ConferenceBaltic Sea Conference on Computing Education Research 2017
    Abbreviated titleKoli Calling 2017
    CountryFinland
    CityKoli
    Period16/11/1719/11/17
    Internet address

    Keywords

    • Data analysis.
    • MOOC
    • Online discussion
    • Programming

    Cite this

    Morgan, M., Nylén, A., Butler, M., Eckerdal, A., Thota, N., & Kinnunen, P. (2017). Examining manual and semi-automated methods of analysing MOOC data for computing education. In C. S. Montero, & M. Joy (Eds.), Proceedings: 17th Koli Calling Conference on Computing Education Research - Koli Calling 2017 (pp. 153-157). New York NY USA: Association for Computing Machinery (ACM). https://doi.org/10.1145/3141880.3141887
    Morgan, Michael ; Nylén, Aletta ; Butler, Matthew ; Eckerdal, Anna ; Thota, Neena ; Kinnunen, Päivi. / Examining manual and semi-automated methods of analysing MOOC data for computing education. Proceedings: 17th Koli Calling Conference on Computing Education Research - Koli Calling 2017. editor / Calkin Suero Montero ; Mike Joy. New York NY USA : Association for Computing Machinery (ACM), 2017. pp. 153-157
    @inproceedings{12b4d9993e634c2d939ad5ba3d5757fa,
    title = "Examining manual and semi-automated methods of analysing MOOC data for computing education",
    abstract = "We examine a semi-automated approach to the analysis of data from MOOC discussion forums. Previous research had analysed a sample of discussion forum data and developed a manual analysis framework, however this process can be very time consuming, especially given the class size of some online courses. Therefore it is important to investigate appropriate and automated analysis techniques to improve timeliness of analysis and to reveal the topics that emerge from a semi-automated process. An analysis of a data set from a coding MOOC in 2015 using the automated Structural Topic Modeling (STM) technique in R is described and contrasted against a manual analysis conducted on a segment of data from the same course in 2014. The types of analyses available and the relevance to computing education research is highlighted, with a focus on providing a discussion of the contrasting capabilities of each approach. The aim is to enable computing education researchers to assess the relevance of these techniques for further work.",
    keywords = "Data analysis., MOOC, Online discussion, Programming",
    author = "Michael Morgan and Aletta Nyl{\'e}n and Matthew Butler and Anna Eckerdal and Neena Thota and P{\"a}ivi Kinnunen",
    year = "2017",
    month = "11",
    day = "16",
    doi = "10.1145/3141880.3141887",
    language = "English",
    isbn = "9781450353014",
    pages = "153--157",
    editor = "Montero, {Calkin Suero} and Mike Joy",
    booktitle = "Proceedings",
    publisher = "Association for Computing Machinery (ACM)",
    address = "United States",

    }

    Morgan, M, Nylén, A, Butler, M, Eckerdal, A, Thota, N & Kinnunen, P 2017, Examining manual and semi-automated methods of analysing MOOC data for computing education. in CS Montero & M Joy (eds), Proceedings: 17th Koli Calling Conference on Computing Education Research - Koli Calling 2017. Association for Computing Machinery (ACM), New York NY USA, pp. 153-157, Baltic Sea Conference on Computing Education Research 2017, Koli, Finland, 16/11/17. https://doi.org/10.1145/3141880.3141887

    Examining manual and semi-automated methods of analysing MOOC data for computing education. / Morgan, Michael; Nylén, Aletta; Butler, Matthew; Eckerdal, Anna; Thota, Neena; Kinnunen, Päivi.

    Proceedings: 17th Koli Calling Conference on Computing Education Research - Koli Calling 2017. ed. / Calkin Suero Montero; Mike Joy. New York NY USA : Association for Computing Machinery (ACM), 2017. p. 153-157.

    Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

    TY - GEN

    T1 - Examining manual and semi-automated methods of analysing MOOC data for computing education

    AU - Morgan, Michael

    AU - Nylén, Aletta

    AU - Butler, Matthew

    AU - Eckerdal, Anna

    AU - Thota, Neena

    AU - Kinnunen, Päivi

    PY - 2017/11/16

    Y1 - 2017/11/16

    N2 - We examine a semi-automated approach to the analysis of data from MOOC discussion forums. Previous research had analysed a sample of discussion forum data and developed a manual analysis framework, however this process can be very time consuming, especially given the class size of some online courses. Therefore it is important to investigate appropriate and automated analysis techniques to improve timeliness of analysis and to reveal the topics that emerge from a semi-automated process. An analysis of a data set from a coding MOOC in 2015 using the automated Structural Topic Modeling (STM) technique in R is described and contrasted against a manual analysis conducted on a segment of data from the same course in 2014. The types of analyses available and the relevance to computing education research is highlighted, with a focus on providing a discussion of the contrasting capabilities of each approach. The aim is to enable computing education researchers to assess the relevance of these techniques for further work.

    AB - We examine a semi-automated approach to the analysis of data from MOOC discussion forums. Previous research had analysed a sample of discussion forum data and developed a manual analysis framework, however this process can be very time consuming, especially given the class size of some online courses. Therefore it is important to investigate appropriate and automated analysis techniques to improve timeliness of analysis and to reveal the topics that emerge from a semi-automated process. An analysis of a data set from a coding MOOC in 2015 using the automated Structural Topic Modeling (STM) technique in R is described and contrasted against a manual analysis conducted on a segment of data from the same course in 2014. The types of analyses available and the relevance to computing education research is highlighted, with a focus on providing a discussion of the contrasting capabilities of each approach. The aim is to enable computing education researchers to assess the relevance of these techniques for further work.

    KW - Data analysis.

    KW - MOOC

    KW - Online discussion

    KW - Programming

    UR - http://www.scopus.com/inward/record.url?scp=85040698000&partnerID=8YFLogxK

    U2 - 10.1145/3141880.3141887

    DO - 10.1145/3141880.3141887

    M3 - Conference Paper

    SN - 9781450353014

    SP - 153

    EP - 157

    BT - Proceedings

    A2 - Montero, Calkin Suero

    A2 - Joy, Mike

    PB - Association for Computing Machinery (ACM)

    CY - New York NY USA

    ER -

    Morgan M, Nylén A, Butler M, Eckerdal A, Thota N, Kinnunen P. Examining manual and semi-automated methods of analysing MOOC data for computing education. In Montero CS, Joy M, editors, Proceedings: 17th Koli Calling Conference on Computing Education Research - Koli Calling 2017. New York NY USA: Association for Computing Machinery (ACM). 2017. p. 153-157 https://doi.org/10.1145/3141880.3141887