Living systematic reviews

3. Statistical methods for updating meta-analyses

Mark Simmonds, Georgia Salanti, Joanne McKenzie, Julian Elliott

Research output: Contribution to journalArticleOtherpeer-review

21 Citations (Scopus)

Abstract

A living systematic review (LSR) should keep the review current as new research evidence emerges. Any meta-analyses included in the review will also need updating as new material is identified. If the aim of the review is solely to present the best current evidence standard meta-analysis may be sufficient, provided reviewers are aware that results may change at later updates. If the review is used in a decision-making context, more caution may be needed. When using standard meta-analysis methods, the chance of incorrectly concluding that any updated meta-analysis is statistically significant when there is no effect (the type I error) increases rapidly as more updates are performed. Inaccurate estimation of any heterogeneity across studies may also lead to inappropriate conclusions. This paper considers four methods to avoid some of these statistical problems when updating meta-analyses: two methods, that is, law of the iterated logarithm and the Shuster method control primarily for inflation of type I error and two other methods, that is, trial sequential analysis and sequential meta-analysis control for type I and II errors (failing to detect a genuine effect) and take account of heterogeneity. This paper compares the methods and considers how they could be applied to LSRs.

Original languageEnglish
Pages (from-to)38-46
Number of pages9
JournalJournal of Clinical Epidemiology
Volume91
DOIs
Publication statusPublished - Nov 2017

Keywords

  • Heterogeneity
  • Living systematic review
  • Meta-analysis
  • Type I error
  • Type II error

Cite this

@article{a50beeb0ec384c5bb936a49cffb501e9,
title = "Living systematic reviews: 3. Statistical methods for updating meta-analyses",
abstract = "A living systematic review (LSR) should keep the review current as new research evidence emerges. Any meta-analyses included in the review will also need updating as new material is identified. If the aim of the review is solely to present the best current evidence standard meta-analysis may be sufficient, provided reviewers are aware that results may change at later updates. If the review is used in a decision-making context, more caution may be needed. When using standard meta-analysis methods, the chance of incorrectly concluding that any updated meta-analysis is statistically significant when there is no effect (the type I error) increases rapidly as more updates are performed. Inaccurate estimation of any heterogeneity across studies may also lead to inappropriate conclusions. This paper considers four methods to avoid some of these statistical problems when updating meta-analyses: two methods, that is, law of the iterated logarithm and the Shuster method control primarily for inflation of type I error and two other methods, that is, trial sequential analysis and sequential meta-analysis control for type I and II errors (failing to detect a genuine effect) and take account of heterogeneity. This paper compares the methods and considers how they could be applied to LSRs.",
keywords = "Heterogeneity, Living systematic review, Meta-analysis, Type I error, Type II error",
author = "Mark Simmonds and Georgia Salanti and Joanne McKenzie and Julian Elliott",
year = "2017",
month = "11",
doi = "10.1016/j.jclinepi.2017.08.008",
language = "English",
volume = "91",
pages = "38--46",
journal = "Journal of Clinical Epidemiology",
issn = "0895-4356",
publisher = "Elsevier",

}

Living systematic reviews : 3. Statistical methods for updating meta-analyses. / Simmonds, Mark; Salanti, Georgia; McKenzie, Joanne; Elliott, Julian.

In: Journal of Clinical Epidemiology, Vol. 91, 11.2017, p. 38-46.

Research output: Contribution to journalArticleOtherpeer-review

TY - JOUR

T1 - Living systematic reviews

T2 - 3. Statistical methods for updating meta-analyses

AU - Simmonds, Mark

AU - Salanti, Georgia

AU - McKenzie, Joanne

AU - Elliott, Julian

PY - 2017/11

Y1 - 2017/11

N2 - A living systematic review (LSR) should keep the review current as new research evidence emerges. Any meta-analyses included in the review will also need updating as new material is identified. If the aim of the review is solely to present the best current evidence standard meta-analysis may be sufficient, provided reviewers are aware that results may change at later updates. If the review is used in a decision-making context, more caution may be needed. When using standard meta-analysis methods, the chance of incorrectly concluding that any updated meta-analysis is statistically significant when there is no effect (the type I error) increases rapidly as more updates are performed. Inaccurate estimation of any heterogeneity across studies may also lead to inappropriate conclusions. This paper considers four methods to avoid some of these statistical problems when updating meta-analyses: two methods, that is, law of the iterated logarithm and the Shuster method control primarily for inflation of type I error and two other methods, that is, trial sequential analysis and sequential meta-analysis control for type I and II errors (failing to detect a genuine effect) and take account of heterogeneity. This paper compares the methods and considers how they could be applied to LSRs.

AB - A living systematic review (LSR) should keep the review current as new research evidence emerges. Any meta-analyses included in the review will also need updating as new material is identified. If the aim of the review is solely to present the best current evidence standard meta-analysis may be sufficient, provided reviewers are aware that results may change at later updates. If the review is used in a decision-making context, more caution may be needed. When using standard meta-analysis methods, the chance of incorrectly concluding that any updated meta-analysis is statistically significant when there is no effect (the type I error) increases rapidly as more updates are performed. Inaccurate estimation of any heterogeneity across studies may also lead to inappropriate conclusions. This paper considers four methods to avoid some of these statistical problems when updating meta-analyses: two methods, that is, law of the iterated logarithm and the Shuster method control primarily for inflation of type I error and two other methods, that is, trial sequential analysis and sequential meta-analysis control for type I and II errors (failing to detect a genuine effect) and take account of heterogeneity. This paper compares the methods and considers how they could be applied to LSRs.

KW - Heterogeneity

KW - Living systematic review

KW - Meta-analysis

KW - Type I error

KW - Type II error

UR - http://www.scopus.com/inward/record.url?scp=85028983900&partnerID=8YFLogxK

U2 - 10.1016/j.jclinepi.2017.08.008

DO - 10.1016/j.jclinepi.2017.08.008

M3 - Article

VL - 91

SP - 38

EP - 46

JO - Journal of Clinical Epidemiology

JF - Journal of Clinical Epidemiology

SN - 0895-4356

ER -