Validation of data submitted by the treating surgeon in the Victorian Audit of Surgical Mortality

Dylan Hansen, Emma Hansen, Claudia Retegan, Julia Morphet, Charles Barry Beiles

Research output: Contribution to journalArticleResearchpeer-review

1 Citation (Scopus)

Abstract

Background: The Victorian Audit of Surgical Mortality (VASM) seeks to peer review all deaths associated with surgical care in Victoria, Australia. The effectiveness of the VASM as an educational and quality improvement tool is dependent on the accuracy of source data it receives. We aimed to examine the accuracy and quality of source data provided by the treating surgeon for peer review, and the inter-rater concordance level between the external validator findings and the treating surgeon. Methods: Of the 629 cases that completed the VASM audit second-line peer review process over a 4-year period (from 1 July 2012 to 30 June 2016), a total of 32 (5%) were randomly selected for the external validation process. The blinded external validator was impartial to the VASM audit, and was provided only de-identified patient medical records. The analysis for the checked and validated data points and their concordance was determined using Gwet's agreement coefficient, which provides a stable inter-rater reliability coefficient not affected by prevalence and marginal probability. Results: The inter-rater concordance analysis suggested that there is a high level of agreement (82.9% overall) between the treating surgeon and external validator. The use of thromboembolism deterrent stockings was the only variable where agreement was poor (52.4%) with a Gwet score of 0.10 (−0.40 to 0.60). Conclusion: The inter-rater concordance analysis results support the validity of the VASM process, which is dependent on the accuracy of data submitted by the treating surgeon.

Original languageEnglish
Pages (from-to)16-19
Number of pages4
JournalANZ Journal of Surgery
Volume89
Issue number1-2
DOIs
Publication statusPublished - Jan 2019

Keywords

  • clinical audit
  • healthcare quality assurance
  • mortality
  • surgery
  • validity

Cite this

Hansen, Dylan ; Hansen, Emma ; Retegan, Claudia ; Morphet, Julia ; Beiles, Charles Barry. / Validation of data submitted by the treating surgeon in the Victorian Audit of Surgical Mortality. In: ANZ Journal of Surgery. 2019 ; Vol. 89, No. 1-2. pp. 16-19.
@article{37469296dc584ce49069d37159bc6bac,
title = "Validation of data submitted by the treating surgeon in the Victorian Audit of Surgical Mortality",
abstract = "Background: The Victorian Audit of Surgical Mortality (VASM) seeks to peer review all deaths associated with surgical care in Victoria, Australia. The effectiveness of the VASM as an educational and quality improvement tool is dependent on the accuracy of source data it receives. We aimed to examine the accuracy and quality of source data provided by the treating surgeon for peer review, and the inter-rater concordance level between the external validator findings and the treating surgeon. Methods: Of the 629 cases that completed the VASM audit second-line peer review process over a 4-year period (from 1 July 2012 to 30 June 2016), a total of 32 (5{\%}) were randomly selected for the external validation process. The blinded external validator was impartial to the VASM audit, and was provided only de-identified patient medical records. The analysis for the checked and validated data points and their concordance was determined using Gwet's agreement coefficient, which provides a stable inter-rater reliability coefficient not affected by prevalence and marginal probability. Results: The inter-rater concordance analysis suggested that there is a high level of agreement (82.9{\%} overall) between the treating surgeon and external validator. The use of thromboembolism deterrent stockings was the only variable where agreement was poor (52.4{\%}) with a Gwet score of 0.10 (−0.40 to 0.60). Conclusion: The inter-rater concordance analysis results support the validity of the VASM process, which is dependent on the accuracy of data submitted by the treating surgeon.",
keywords = "clinical audit, healthcare quality assurance, mortality, surgery, validity",
author = "Dylan Hansen and Emma Hansen and Claudia Retegan and Julia Morphet and Beiles, {Charles Barry}",
year = "2019",
month = "1",
doi = "10.1111/ans.14910",
language = "English",
volume = "89",
pages = "16--19",
journal = "ANZ Journal of Surgery",
issn = "1445-1433",
publisher = "Wiley-Blackwell",
number = "1-2",

}

Validation of data submitted by the treating surgeon in the Victorian Audit of Surgical Mortality. / Hansen, Dylan; Hansen, Emma; Retegan, Claudia; Morphet, Julia; Beiles, Charles Barry.

In: ANZ Journal of Surgery, Vol. 89, No. 1-2, 01.2019, p. 16-19.

Research output: Contribution to journalArticleResearchpeer-review

TY - JOUR

T1 - Validation of data submitted by the treating surgeon in the Victorian Audit of Surgical Mortality

AU - Hansen, Dylan

AU - Hansen, Emma

AU - Retegan, Claudia

AU - Morphet, Julia

AU - Beiles, Charles Barry

PY - 2019/1

Y1 - 2019/1

N2 - Background: The Victorian Audit of Surgical Mortality (VASM) seeks to peer review all deaths associated with surgical care in Victoria, Australia. The effectiveness of the VASM as an educational and quality improvement tool is dependent on the accuracy of source data it receives. We aimed to examine the accuracy and quality of source data provided by the treating surgeon for peer review, and the inter-rater concordance level between the external validator findings and the treating surgeon. Methods: Of the 629 cases that completed the VASM audit second-line peer review process over a 4-year period (from 1 July 2012 to 30 June 2016), a total of 32 (5%) were randomly selected for the external validation process. The blinded external validator was impartial to the VASM audit, and was provided only de-identified patient medical records. The analysis for the checked and validated data points and their concordance was determined using Gwet's agreement coefficient, which provides a stable inter-rater reliability coefficient not affected by prevalence and marginal probability. Results: The inter-rater concordance analysis suggested that there is a high level of agreement (82.9% overall) between the treating surgeon and external validator. The use of thromboembolism deterrent stockings was the only variable where agreement was poor (52.4%) with a Gwet score of 0.10 (−0.40 to 0.60). Conclusion: The inter-rater concordance analysis results support the validity of the VASM process, which is dependent on the accuracy of data submitted by the treating surgeon.

AB - Background: The Victorian Audit of Surgical Mortality (VASM) seeks to peer review all deaths associated with surgical care in Victoria, Australia. The effectiveness of the VASM as an educational and quality improvement tool is dependent on the accuracy of source data it receives. We aimed to examine the accuracy and quality of source data provided by the treating surgeon for peer review, and the inter-rater concordance level between the external validator findings and the treating surgeon. Methods: Of the 629 cases that completed the VASM audit second-line peer review process over a 4-year period (from 1 July 2012 to 30 June 2016), a total of 32 (5%) were randomly selected for the external validation process. The blinded external validator was impartial to the VASM audit, and was provided only de-identified patient medical records. The analysis for the checked and validated data points and their concordance was determined using Gwet's agreement coefficient, which provides a stable inter-rater reliability coefficient not affected by prevalence and marginal probability. Results: The inter-rater concordance analysis suggested that there is a high level of agreement (82.9% overall) between the treating surgeon and external validator. The use of thromboembolism deterrent stockings was the only variable where agreement was poor (52.4%) with a Gwet score of 0.10 (−0.40 to 0.60). Conclusion: The inter-rater concordance analysis results support the validity of the VASM process, which is dependent on the accuracy of data submitted by the treating surgeon.

KW - clinical audit

KW - healthcare quality assurance

KW - mortality

KW - surgery

KW - validity

UR - http://www.scopus.com/inward/record.url?scp=85057786131&partnerID=8YFLogxK

U2 - 10.1111/ans.14910

DO - 10.1111/ans.14910

M3 - Article

VL - 89

SP - 16

EP - 19

JO - ANZ Journal of Surgery

JF - ANZ Journal of Surgery

SN - 1445-1433

IS - 1-2

ER -