Toward a comprehensive evidence map of overview of systematic review methods

Paper 2 - Risk of bias assessment; Synthesis, presentation and summary of the findings; And assessment of the certainty of the evidence

Research output: Contribution to journalArticleOtherpeer-review

6 Citations (Scopus)

Abstract

Background: Overviews of systematic reviews (SRs) attempt to systematically retrieve and summarise the results of multiple systematic reviews. This is the second of two papers from a study aiming to develop a comprehensive evidence map of the methods used in overviews. Our objectives were to (a) develop a framework of methods for conducting, interpreting and reporting overviews (stage I) - the Methods for Overviews of Reviews (MOoR) framework - and (b) to create an evidence map by mapping studies that have evaluated overview methods to the framework (stage II). In the first paper, we reported findings for the four initial steps of an overview (specification of purpose, objectives and scope; eligibility criteria; search methods; data extraction). In this paper, we report the remaining steps: assessing risk of bias; synthesis, presentation and summary of the findings; and assessing certainty of the evidence arising from the overview. Methods: In stage I, we identified cross-sectional studies, guidance documents and commentaries that described methods proposed for, or used in, overviews. Based on these studies, we developed a framework of possible methods for overviews, categorised by the steps in conducting an overview. Multiple iterations of the framework were discussed and refined by all authors. In stage II, we identified studies evaluating methods and mapped these evaluations to the framework. Results: Forty-two stage I studies described methods relevant to one or more of the latter steps of an overview. Six studies evaluating methods were included in stage II. These mapped to steps involving (i) the assessment of risk of bias (RoB) in SRs (two SRs and three primary studies, all reporting evaluation of RoB tools) and (ii) the synthesis, presentation and summary of the findings (one primary study evaluating methods for measuring overlap). Conclusion: Many methods have been described for use in the latter steps in conducting an overview; however, evaluation and guidance for applying these methods is sparse. The exception is RoB assessment, for which a multitude of tools exist - several with sufficient evaluation and guidance to recommend their use. Evaluation of other methods is required to provide a comprehensive evidence map.

Original languageEnglish
Article number159
Number of pages31
JournalSystematic Reviews
Volume7
Issue number1
DOIs
Publication statusPublished - 12 Oct 2018

Keywords

  • Assessment of risk of bias in systematic reviews
  • Evaluation of methods
  • Evidence map
  • Meta-review
  • Methodology
  • Overview
  • Overview of systematic reviews
  • Review of reviews
  • Systematic review methods
  • Umbrella review

Cite this

@article{35464bdd41e0443387f2b7f0b2a24988,
title = "Toward a comprehensive evidence map of overview of systematic review methods: Paper 2 - Risk of bias assessment; Synthesis, presentation and summary of the findings; And assessment of the certainty of the evidence",
abstract = "Background: Overviews of systematic reviews (SRs) attempt to systematically retrieve and summarise the results of multiple systematic reviews. This is the second of two papers from a study aiming to develop a comprehensive evidence map of the methods used in overviews. Our objectives were to (a) develop a framework of methods for conducting, interpreting and reporting overviews (stage I) - the Methods for Overviews of Reviews (MOoR) framework - and (b) to create an evidence map by mapping studies that have evaluated overview methods to the framework (stage II). In the first paper, we reported findings for the four initial steps of an overview (specification of purpose, objectives and scope; eligibility criteria; search methods; data extraction). In this paper, we report the remaining steps: assessing risk of bias; synthesis, presentation and summary of the findings; and assessing certainty of the evidence arising from the overview. Methods: In stage I, we identified cross-sectional studies, guidance documents and commentaries that described methods proposed for, or used in, overviews. Based on these studies, we developed a framework of possible methods for overviews, categorised by the steps in conducting an overview. Multiple iterations of the framework were discussed and refined by all authors. In stage II, we identified studies evaluating methods and mapped these evaluations to the framework. Results: Forty-two stage I studies described methods relevant to one or more of the latter steps of an overview. Six studies evaluating methods were included in stage II. These mapped to steps involving (i) the assessment of risk of bias (RoB) in SRs (two SRs and three primary studies, all reporting evaluation of RoB tools) and (ii) the synthesis, presentation and summary of the findings (one primary study evaluating methods for measuring overlap). Conclusion: Many methods have been described for use in the latter steps in conducting an overview; however, evaluation and guidance for applying these methods is sparse. The exception is RoB assessment, for which a multitude of tools exist - several with sufficient evaluation and guidance to recommend their use. Evaluation of other methods is required to provide a comprehensive evidence map.",
keywords = "Assessment of risk of bias in systematic reviews, Evaluation of methods, Evidence map, Meta-review, Methodology, Overview, Overview of systematic reviews, Review of reviews, Systematic review methods, Umbrella review",
author = "Carole Lunny and Brennan, {Sue E.} and Steve McDonald and McKenzie, {Joanne E.}",
year = "2018",
month = "10",
day = "12",
doi = "10.1186/s13643-018-0784-8",
language = "English",
volume = "7",
journal = "Systematic Reviews",
issn = "2046-4053",
publisher = "Springer-Verlag London Ltd.",
number = "1",

}

TY - JOUR

T1 - Toward a comprehensive evidence map of overview of systematic review methods

T2 - Paper 2 - Risk of bias assessment; Synthesis, presentation and summary of the findings; And assessment of the certainty of the evidence

AU - Lunny, Carole

AU - Brennan, Sue E.

AU - McDonald, Steve

AU - McKenzie, Joanne E.

PY - 2018/10/12

Y1 - 2018/10/12

N2 - Background: Overviews of systematic reviews (SRs) attempt to systematically retrieve and summarise the results of multiple systematic reviews. This is the second of two papers from a study aiming to develop a comprehensive evidence map of the methods used in overviews. Our objectives were to (a) develop a framework of methods for conducting, interpreting and reporting overviews (stage I) - the Methods for Overviews of Reviews (MOoR) framework - and (b) to create an evidence map by mapping studies that have evaluated overview methods to the framework (stage II). In the first paper, we reported findings for the four initial steps of an overview (specification of purpose, objectives and scope; eligibility criteria; search methods; data extraction). In this paper, we report the remaining steps: assessing risk of bias; synthesis, presentation and summary of the findings; and assessing certainty of the evidence arising from the overview. Methods: In stage I, we identified cross-sectional studies, guidance documents and commentaries that described methods proposed for, or used in, overviews. Based on these studies, we developed a framework of possible methods for overviews, categorised by the steps in conducting an overview. Multiple iterations of the framework were discussed and refined by all authors. In stage II, we identified studies evaluating methods and mapped these evaluations to the framework. Results: Forty-two stage I studies described methods relevant to one or more of the latter steps of an overview. Six studies evaluating methods were included in stage II. These mapped to steps involving (i) the assessment of risk of bias (RoB) in SRs (two SRs and three primary studies, all reporting evaluation of RoB tools) and (ii) the synthesis, presentation and summary of the findings (one primary study evaluating methods for measuring overlap). Conclusion: Many methods have been described for use in the latter steps in conducting an overview; however, evaluation and guidance for applying these methods is sparse. The exception is RoB assessment, for which a multitude of tools exist - several with sufficient evaluation and guidance to recommend their use. Evaluation of other methods is required to provide a comprehensive evidence map.

AB - Background: Overviews of systematic reviews (SRs) attempt to systematically retrieve and summarise the results of multiple systematic reviews. This is the second of two papers from a study aiming to develop a comprehensive evidence map of the methods used in overviews. Our objectives were to (a) develop a framework of methods for conducting, interpreting and reporting overviews (stage I) - the Methods for Overviews of Reviews (MOoR) framework - and (b) to create an evidence map by mapping studies that have evaluated overview methods to the framework (stage II). In the first paper, we reported findings for the four initial steps of an overview (specification of purpose, objectives and scope; eligibility criteria; search methods; data extraction). In this paper, we report the remaining steps: assessing risk of bias; synthesis, presentation and summary of the findings; and assessing certainty of the evidence arising from the overview. Methods: In stage I, we identified cross-sectional studies, guidance documents and commentaries that described methods proposed for, or used in, overviews. Based on these studies, we developed a framework of possible methods for overviews, categorised by the steps in conducting an overview. Multiple iterations of the framework were discussed and refined by all authors. In stage II, we identified studies evaluating methods and mapped these evaluations to the framework. Results: Forty-two stage I studies described methods relevant to one or more of the latter steps of an overview. Six studies evaluating methods were included in stage II. These mapped to steps involving (i) the assessment of risk of bias (RoB) in SRs (two SRs and three primary studies, all reporting evaluation of RoB tools) and (ii) the synthesis, presentation and summary of the findings (one primary study evaluating methods for measuring overlap). Conclusion: Many methods have been described for use in the latter steps in conducting an overview; however, evaluation and guidance for applying these methods is sparse. The exception is RoB assessment, for which a multitude of tools exist - several with sufficient evaluation and guidance to recommend their use. Evaluation of other methods is required to provide a comprehensive evidence map.

KW - Assessment of risk of bias in systematic reviews

KW - Evaluation of methods

KW - Evidence map

KW - Meta-review

KW - Methodology

KW - Overview

KW - Overview of systematic reviews

KW - Review of reviews

KW - Systematic review methods

KW - Umbrella review

UR - http://www.scopus.com/inward/record.url?scp=85054875571&partnerID=8YFLogxK

U2 - 10.1186/s13643-018-0784-8

DO - 10.1186/s13643-018-0784-8

M3 - Article

VL - 7

JO - Systematic Reviews

JF - Systematic Reviews

SN - 2046-4053

IS - 1

M1 - 159

ER -