Abstract
The process of labeling, categorizing, or otherwise annotating data—or coding in the computer-supported collaborative learning (CSCL) literature—is a fundamental process in CSCL research. It is the process by which researchers identify salient properties about segments of CSCL data: What they are, what they contain, or what they mean. Coding, like all processes in research, is subject to error. To reduce the potential impact of coding error, CSCL researchers typically measure inter-rater reliability (IRR). However, there is no extant method to determine what level of IRR would invalidate a CSCL result or model. One way of assessing the potential impact of such inaccuracies is by conducting sensitivity analyses, which measure the level of error that would need to be present in the data to invalidate a given inference. This paper introduces a new method for conducting sensitivity analyses in CSCL: The Binary Replicate Test.
Original language | English |
---|---|
Title of host publication | 13th International Conference on Computer Supported Collaborative Learning |
Subtitle of host publication | Conference Proceedings |
Editors | Kristine Lund, Gerald P. Niccolai, Elise Lavoue, Cindy Hmelo-Silver, Gahgene Gweon, Michael Baker |
Place of Publication | Lyon France |
Publisher | International Society of the Learning Sciences (ISLS) |
Pages | 328-335 |
Number of pages | 8 |
ISBN (Electronic) | 9781732467231 |
Publication status | Published - 2019 |
Externally published | Yes |
Event | Computer Supported Collaborative Learning 2019 - Lyon, France Duration: 17 Jun 2019 → 21 Jun 2019 Conference number: 13th https://www.cscl2019.com/ https://www.isls.org/cscl/2019/www.cscl2019.com/index.html (Proceedings) |
Publication series
Name | Computer-Supported Collaborative Learning Conference, CSCL |
---|---|
Publisher | International Society of the Learning Sciences, Inc. |
Volume | 1 |
ISSN (Electronic) | 1573-4552 |
Conference
Conference | Computer Supported Collaborative Learning 2019 |
---|---|
Abbreviated title | CSCL 2019 |
Country/Territory | France |
City | Lyon |
Period | 17/06/19 → 21/06/19 |
Internet address |