Inferential statistics in computing education research: a methodological review

Kate Sanders, Judy Sheard, Brett A. Becker, Anna Eckerdal, Sally Hamouda, Simon

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

Abstract

The goal of most computing education research is to effect positive change in how computing is taught and learned. Statistical techniques are one important tool for achieving this goal. In this paper we report on an analysis of ICER papers that use inferential statistics. We present the most commonly used techniques; an overview of the techniques the ICER community has used over its first 14 years of papers, grouped according to the purpose of the technique; and a detailed analysis of three of the most commonly used techniques (t-test, chi-squared test, and Mann-Whitney-Wilcoxon). We identify common flaws in reporting and give examples of papers where statistics are reported well. In sum, the paper draws a picture of the use of inferential statistics by the ICER community. This picture is intended to help orient researchers who are new to the use of statistics in computing education research and to encourage reflection by the ICER community on how it uses statistics and how it can improve that use.

Original languageEnglish
Title of host publicationProceedings of the 2019 ACM Conference on International Computing Education Research
EditorsRobert McCartney, Anthony Robins
Place of PublicationNew York NY USA
PublisherAssociation for Computing Machinery (ACM)
Pages177-185
Number of pages9
ISBN (Electronic)9781450361859
DOIs
Publication statusPublished - 2019
EventInternational Computing Education Research Conference 2019 - Toronto, Canada
Duration: 12 Aug 201914 Aug 2019
Conference number: 15th
https://icer.acm.org/

Conference

ConferenceInternational Computing Education Research Conference 2019
Abbreviated titleICER 2019
CountryCanada
CityToronto
Period12/08/1914/08/19
Internet address

Keywords

  • Alphabet soup
  • Empirical research
  • Methodology
  • Reporting
  • Statistical significance
  • Statistics

Cite this

Sanders, K., Sheard, J., Becker, B. A., Eckerdal, A., Hamouda, S., & Simon (2019). Inferential statistics in computing education research: a methodological review. In R. McCartney, & A. Robins (Eds.), Proceedings of the 2019 ACM Conference on International Computing Education Research (pp. 177-185). Association for Computing Machinery (ACM). https://doi.org/10.1145/3291279.3339408