Validation of a classification system for causes of death in critical care: An assessment of inter-rater reliability

Elliott Ridgeon, Rinaldo Bellomo, John Myburgh, Manoj Saxena, Mark Weatherall, Rahi Jahan, Dilshan Arawwawala, Stephen Bell, Warwick Butt, Julie Camsooksai, Coralie Carle, Andrew Cheng, Emanuel Cirstea, Jeremy Cohen, Julius Cranshaw, Anthony Delaney, Glenn Eastwood, Suzanne Eliott, Uwe Franke, Dashiell Gantner & 3 others Lynette Newby, Yahya Shehabi, ICU-DECLARE Investigators

Research output: Contribution to journalArticleResearchpeer-review

Abstract

Objective: Trials in critical care have previously used unvalidated systems to classify cause of death. We aimed to provide initial validation of a method to classify cause of death in intensive care unit patients. Design, setting and participants: One hundred case scenarios of patients who died in an ICU were presented online to raters, who were asked to select a proximate and an underlying cause of death for each, using the ICU Deaths Classification and Reason (ICU-DECLARE) system. We evaluated two methods of categorising proximate cause of death (designated Lists A and B) and one method of categorising underlying cause of death. Raters were ICU specialists and research coordinators from Australia, New Zealand and the United Kingdom. Main outcome measures: Inter-rater reliability, as measured by the Fleiss multirater kappa, and the median proportion of raters choosing the most likely diagnosis (defined as the most popular classification choice in each case). Results: Across all raters and cases, for proximate cause of death List A, kappa was 0.54 (95% Cl, 0.49–0.60), and for proximate cause of death List B, kappa was 0.58 (95% Cl, 0.53–0.63). For the underlying cause of death, kappa was 0.48 (95% Cl, 0.44–0.53). The median proportion of raters choosing the most likely diagnosis for proximate cause of death, List A, was 77.5% (interquartile range [IQR], 60.0%–93.8%), and the median proportion choosing the most likely diagnosis for proximate cause of death, List B, was 82.5% (IQR, 60.0%–92.5%). The median proportion choosing the most likely diagnosis for underlying cause was 65.0% (IQR, 50.0%–81.3%). Kappa and median agreement were similar between countries. ICU specialists showed higher kappa and median agreement than research coordinators. Conclusions: The ICU-DECLARE system allowed ICU doctors to classify the proximate cause of death of patients who died in the ICU with substantial reliability.

Original languageEnglish
Pages (from-to)50-54
Number of pages5
JournalCritical Care and Resuscitation
Volume18
Issue number1
Publication statusPublished - Mar 2016

Cite this

Ridgeon, Elliott ; Bellomo, Rinaldo ; Myburgh, John ; Saxena, Manoj ; Weatherall, Mark ; Jahan, Rahi ; Arawwawala, Dilshan ; Bell, Stephen ; Butt, Warwick ; Camsooksai, Julie ; Carle, Coralie ; Cheng, Andrew ; Cirstea, Emanuel ; Cohen, Jeremy ; Cranshaw, Julius ; Delaney, Anthony ; Eastwood, Glenn ; Eliott, Suzanne ; Franke, Uwe ; Gantner, Dashiell ; Newby, Lynette ; Shehabi, Yahya ; ICU-DECLARE Investigators. / Validation of a classification system for causes of death in critical care : An assessment of inter-rater reliability. In: Critical Care and Resuscitation. 2016 ; Vol. 18, No. 1. pp. 50-54.
@article{06321c5a26864191a48e372761856526,
title = "Validation of a classification system for causes of death in critical care: An assessment of inter-rater reliability",
abstract = "Objective: Trials in critical care have previously used unvalidated systems to classify cause of death. We aimed to provide initial validation of a method to classify cause of death in intensive care unit patients. Design, setting and participants: One hundred case scenarios of patients who died in an ICU were presented online to raters, who were asked to select a proximate and an underlying cause of death for each, using the ICU Deaths Classification and Reason (ICU-DECLARE) system. We evaluated two methods of categorising proximate cause of death (designated Lists A and B) and one method of categorising underlying cause of death. Raters were ICU specialists and research coordinators from Australia, New Zealand and the United Kingdom. Main outcome measures: Inter-rater reliability, as measured by the Fleiss multirater kappa, and the median proportion of raters choosing the most likely diagnosis (defined as the most popular classification choice in each case). Results: Across all raters and cases, for proximate cause of death List A, kappa was 0.54 (95{\%} Cl, 0.49–0.60), and for proximate cause of death List B, kappa was 0.58 (95{\%} Cl, 0.53–0.63). For the underlying cause of death, kappa was 0.48 (95{\%} Cl, 0.44–0.53). The median proportion of raters choosing the most likely diagnosis for proximate cause of death, List A, was 77.5{\%} (interquartile range [IQR], 60.0{\%}–93.8{\%}), and the median proportion choosing the most likely diagnosis for proximate cause of death, List B, was 82.5{\%} (IQR, 60.0{\%}–92.5{\%}). The median proportion choosing the most likely diagnosis for underlying cause was 65.0{\%} (IQR, 50.0{\%}–81.3{\%}). Kappa and median agreement were similar between countries. ICU specialists showed higher kappa and median agreement than research coordinators. Conclusions: The ICU-DECLARE system allowed ICU doctors to classify the proximate cause of death of patients who died in the ICU with substantial reliability.",
author = "Elliott Ridgeon and Rinaldo Bellomo and John Myburgh and Manoj Saxena and Mark Weatherall and Rahi Jahan and Dilshan Arawwawala and Stephen Bell and Warwick Butt and Julie Camsooksai and Coralie Carle and Andrew Cheng and Emanuel Cirstea and Jeremy Cohen and Julius Cranshaw and Anthony Delaney and Glenn Eastwood and Suzanne Eliott and Uwe Franke and Dashiell Gantner and Lynette Newby and Yahya Shehabi and {ICU-DECLARE Investigators}",
year = "2016",
month = "3",
language = "English",
volume = "18",
pages = "50--54",
journal = "Critical Care and Resuscitation",
issn = "1441-2772",
publisher = "Australasian Medical Publishing Co. Pty Ltd. (AMPCo)",
number = "1",

}

Ridgeon, E, Bellomo, R, Myburgh, J, Saxena, M, Weatherall, M, Jahan, R, Arawwawala, D, Bell, S, Butt, W, Camsooksai, J, Carle, C, Cheng, A, Cirstea, E, Cohen, J, Cranshaw, J, Delaney, A, Eastwood, G, Eliott, S, Franke, U, Gantner, D, Newby, L, Shehabi, Y & ICU-DECLARE Investigators 2016, 'Validation of a classification system for causes of death in critical care: An assessment of inter-rater reliability' Critical Care and Resuscitation, vol. 18, no. 1, pp. 50-54.

Validation of a classification system for causes of death in critical care : An assessment of inter-rater reliability. / Ridgeon, Elliott; Bellomo, Rinaldo; Myburgh, John; Saxena, Manoj; Weatherall, Mark; Jahan, Rahi; Arawwawala, Dilshan; Bell, Stephen; Butt, Warwick; Camsooksai, Julie; Carle, Coralie; Cheng, Andrew; Cirstea, Emanuel; Cohen, Jeremy; Cranshaw, Julius; Delaney, Anthony; Eastwood, Glenn; Eliott, Suzanne; Franke, Uwe; Gantner, Dashiell; Newby, Lynette; Shehabi, Yahya; ICU-DECLARE Investigators.

In: Critical Care and Resuscitation, Vol. 18, No. 1, 03.2016, p. 50-54.

Research output: Contribution to journalArticleResearchpeer-review

TY - JOUR

T1 - Validation of a classification system for causes of death in critical care

T2 - An assessment of inter-rater reliability

AU - Ridgeon, Elliott

AU - Bellomo, Rinaldo

AU - Myburgh, John

AU - Saxena, Manoj

AU - Weatherall, Mark

AU - Jahan, Rahi

AU - Arawwawala, Dilshan

AU - Bell, Stephen

AU - Butt, Warwick

AU - Camsooksai, Julie

AU - Carle, Coralie

AU - Cheng, Andrew

AU - Cirstea, Emanuel

AU - Cohen, Jeremy

AU - Cranshaw, Julius

AU - Delaney, Anthony

AU - Eastwood, Glenn

AU - Eliott, Suzanne

AU - Franke, Uwe

AU - Gantner, Dashiell

AU - Newby, Lynette

AU - Shehabi, Yahya

AU - ICU-DECLARE Investigators

PY - 2016/3

Y1 - 2016/3

N2 - Objective: Trials in critical care have previously used unvalidated systems to classify cause of death. We aimed to provide initial validation of a method to classify cause of death in intensive care unit patients. Design, setting and participants: One hundred case scenarios of patients who died in an ICU were presented online to raters, who were asked to select a proximate and an underlying cause of death for each, using the ICU Deaths Classification and Reason (ICU-DECLARE) system. We evaluated two methods of categorising proximate cause of death (designated Lists A and B) and one method of categorising underlying cause of death. Raters were ICU specialists and research coordinators from Australia, New Zealand and the United Kingdom. Main outcome measures: Inter-rater reliability, as measured by the Fleiss multirater kappa, and the median proportion of raters choosing the most likely diagnosis (defined as the most popular classification choice in each case). Results: Across all raters and cases, for proximate cause of death List A, kappa was 0.54 (95% Cl, 0.49–0.60), and for proximate cause of death List B, kappa was 0.58 (95% Cl, 0.53–0.63). For the underlying cause of death, kappa was 0.48 (95% Cl, 0.44–0.53). The median proportion of raters choosing the most likely diagnosis for proximate cause of death, List A, was 77.5% (interquartile range [IQR], 60.0%–93.8%), and the median proportion choosing the most likely diagnosis for proximate cause of death, List B, was 82.5% (IQR, 60.0%–92.5%). The median proportion choosing the most likely diagnosis for underlying cause was 65.0% (IQR, 50.0%–81.3%). Kappa and median agreement were similar between countries. ICU specialists showed higher kappa and median agreement than research coordinators. Conclusions: The ICU-DECLARE system allowed ICU doctors to classify the proximate cause of death of patients who died in the ICU with substantial reliability.

AB - Objective: Trials in critical care have previously used unvalidated systems to classify cause of death. We aimed to provide initial validation of a method to classify cause of death in intensive care unit patients. Design, setting and participants: One hundred case scenarios of patients who died in an ICU were presented online to raters, who were asked to select a proximate and an underlying cause of death for each, using the ICU Deaths Classification and Reason (ICU-DECLARE) system. We evaluated two methods of categorising proximate cause of death (designated Lists A and B) and one method of categorising underlying cause of death. Raters were ICU specialists and research coordinators from Australia, New Zealand and the United Kingdom. Main outcome measures: Inter-rater reliability, as measured by the Fleiss multirater kappa, and the median proportion of raters choosing the most likely diagnosis (defined as the most popular classification choice in each case). Results: Across all raters and cases, for proximate cause of death List A, kappa was 0.54 (95% Cl, 0.49–0.60), and for proximate cause of death List B, kappa was 0.58 (95% Cl, 0.53–0.63). For the underlying cause of death, kappa was 0.48 (95% Cl, 0.44–0.53). The median proportion of raters choosing the most likely diagnosis for proximate cause of death, List A, was 77.5% (interquartile range [IQR], 60.0%–93.8%), and the median proportion choosing the most likely diagnosis for proximate cause of death, List B, was 82.5% (IQR, 60.0%–92.5%). The median proportion choosing the most likely diagnosis for underlying cause was 65.0% (IQR, 50.0%–81.3%). Kappa and median agreement were similar between countries. ICU specialists showed higher kappa and median agreement than research coordinators. Conclusions: The ICU-DECLARE system allowed ICU doctors to classify the proximate cause of death of patients who died in the ICU with substantial reliability.

UR - http://www.scopus.com/inward/record.url?scp=84983001574&partnerID=8YFLogxK

M3 - Article

VL - 18

SP - 50

EP - 54

JO - Critical Care and Resuscitation

JF - Critical Care and Resuscitation

SN - 1441-2772

IS - 1

ER -