Towards automated content analysis of discussion transcripts: a cognitive presence case

Vitomir Kovanovíc, Srécko Joksimovíc, Zak Waters, Dragan Gaševíc, Kirsty Kitto, Marek Hatala, George Siemens

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

38 Citations (Scopus)

Abstract

In this paper, we present the results of an exploratory study that examined the problem of automating content analysis of student online discussion transcripts. We looked at the problem of coding discussion transcripts for the levels of cognitive presence, one of the three main constructs in the Community of Inquiry (CoI) model of distance education. Using Coh-Metrix and LIWC features, together with a set of custom features developed to capture discussion context, we developed a random forest classification system that achieved 70.3% classification accuracy and 0.63 Cohen's kappa, which is significantly higher than values reported in the previous studies. Besides improvement in classification accuracy, the developed system is also less sensitive to overfitting as it uses only 205 classification features, which is around 100 times less features than in similar systems based on bag-of-words features. We also provide an overview of the classification features most indicative of the different phases of cognitive presence that gives an additional insights into the nature of cognitive presence learning cycle. Overall, our results show great potential of the proposed approach, with an added benefit of providing further characterization of the cognitive presence coding scheme.

Original languageEnglish
Title of host publicationLAK '16 Conference Proceedings
Subtitle of host publicationThe Sixth International Learning Analytics & Knowledge Conference: Enhancing Impact: Convergence of Communities for Grounding, Implementation, and Validation
EditorsShane Dawson, Hendrik Drachsler, Carolyn Penstein Rosé
Place of PublicationNew York NY USA
PublisherAssociation for Computing Machinery (ACM)
Pages15-24
Number of pages10
ISBN (Electronic)9781450341905
DOIs
Publication statusPublished - 2016
Externally publishedYes
EventInternational Learning Analytics & Knowledge Conference 2016 - University of Edinburgh, Edinburgh, United Kingdom
Duration: 25 Apr 201629 Apr 2016
Conference number: 6th
http://lak16.solaresearch.org/

Conference

ConferenceInternational Learning Analytics & Knowledge Conference 2016
Abbreviated titleLAK 2016
CountryUnited Kingdom
CityEdinburgh
Period25/04/1629/04/16
Internet address

Keywords

  • Community of inquiry (coi) model
  • Content analysis
  • Content analytics
  • Online discussions
  • Text classification

Cite this