Overviews of reviews incompletely report methods for handling overlapping, discordant, and problematic data

Research output: Contribution to journalReview ArticleResearchpeer-review

3 Citations (Scopus)

Abstract

Objectives: The aim of the study was to assess the completeness of reporting of methods in overviews. Study Design and Setting: Assessment of the adequacy of reporting of methods in a random sample of 50 overviews was based on a published framework of methods for conducting overviews. Descriptive summary statistics were presented. Results: We screened 848 randomly selected abstracts to obtain the required 50 overviews. Overviews included a median of 13 (interquartile range 7–32) systematic reviews (SRs), 22% reported working from a protocol, 36% reported using reporting standards (e.g., Preferred Reporting Items for Systematic Reviews and Meta-Analyses), and 34% reported using methodological guidance (e.g., Cochrane Handbook). Methods common to both overviews and SRs of primary studies were reported in majority of overviews (e.g., 56% framed the overview question by Population, Intervention(s), Comparison(s), Outcome(s) [PICO] elements; 44% reported eligibility criteria based on PICO, and 76% reported assessing the risk of bias of SRs), except for methods for summarizing evidence (20%) or statistical synthesis (26%). A minority reported methods for handling unique aspects of overviews (e.g., overlap in the primary studies [30%], discrepant or missing data [14%], and discordant results/conclusions across reviews [20%]). Conclusion: Reporting of methods unique to overviews requires improvement. Our findings provide a benchmark of the completeness of reporting and may inform guidance on the conduct and reporting of overviews.

Original languageEnglish
Pages (from-to)69-85
Number of pages17
JournalJournal of Clinical Epidemiology
Volume118
DOIs
Publication statusPublished - 1 Feb 2020

Keywords

  • Meta-review
  • Overview methods
  • Overviews of systematic reviews
  • Reporting
  • Systematic review methods
  • Umbrella

Cite this

@article{141ca04a62514aab8f5d37f9446d4da2,
title = "Overviews of reviews incompletely report methods for handling overlapping, discordant, and problematic data",
abstract = "Objectives: The aim of the study was to assess the completeness of reporting of methods in overviews. Study Design and Setting: Assessment of the adequacy of reporting of methods in a random sample of 50 overviews was based on a published framework of methods for conducting overviews. Descriptive summary statistics were presented. Results: We screened 848 randomly selected abstracts to obtain the required 50 overviews. Overviews included a median of 13 (interquartile range 7–32) systematic reviews (SRs), 22{\%} reported working from a protocol, 36{\%} reported using reporting standards (e.g., Preferred Reporting Items for Systematic Reviews and Meta-Analyses), and 34{\%} reported using methodological guidance (e.g., Cochrane Handbook). Methods common to both overviews and SRs of primary studies were reported in majority of overviews (e.g., 56{\%} framed the overview question by Population, Intervention(s), Comparison(s), Outcome(s) [PICO] elements; 44{\%} reported eligibility criteria based on PICO, and 76{\%} reported assessing the risk of bias of SRs), except for methods for summarizing evidence (20{\%}) or statistical synthesis (26{\%}). A minority reported methods for handling unique aspects of overviews (e.g., overlap in the primary studies [30{\%}], discrepant or missing data [14{\%}], and discordant results/conclusions across reviews [20{\%}]). Conclusion: Reporting of methods unique to overviews requires improvement. Our findings provide a benchmark of the completeness of reporting and may inform guidance on the conduct and reporting of overviews.",
keywords = "Meta-review, Overview methods, Overviews of systematic reviews, Reporting, Systematic review methods, Umbrella",
author = "Carole Lunny and Brennan, {Sue E.} and Jane Reid and Steve McDonald and McKenzie, {Joanne E.}",
year = "2020",
month = "2",
day = "1",
doi = "10.1016/j.jclinepi.2019.09.025",
language = "English",
volume = "118",
pages = "69--85",
journal = "Journal of Clinical Epidemiology",
issn = "0895-4356",
publisher = "Elsevier",

}

Overviews of reviews incompletely report methods for handling overlapping, discordant, and problematic data. / Lunny, Carole; Brennan, Sue E.; Reid, Jane; McDonald, Steve; McKenzie, Joanne E.

In: Journal of Clinical Epidemiology, Vol. 118, 01.02.2020, p. 69-85.

Research output: Contribution to journalReview ArticleResearchpeer-review

TY - JOUR

T1 - Overviews of reviews incompletely report methods for handling overlapping, discordant, and problematic data

AU - Lunny, Carole

AU - Brennan, Sue E.

AU - Reid, Jane

AU - McDonald, Steve

AU - McKenzie, Joanne E.

PY - 2020/2/1

Y1 - 2020/2/1

N2 - Objectives: The aim of the study was to assess the completeness of reporting of methods in overviews. Study Design and Setting: Assessment of the adequacy of reporting of methods in a random sample of 50 overviews was based on a published framework of methods for conducting overviews. Descriptive summary statistics were presented. Results: We screened 848 randomly selected abstracts to obtain the required 50 overviews. Overviews included a median of 13 (interquartile range 7–32) systematic reviews (SRs), 22% reported working from a protocol, 36% reported using reporting standards (e.g., Preferred Reporting Items for Systematic Reviews and Meta-Analyses), and 34% reported using methodological guidance (e.g., Cochrane Handbook). Methods common to both overviews and SRs of primary studies were reported in majority of overviews (e.g., 56% framed the overview question by Population, Intervention(s), Comparison(s), Outcome(s) [PICO] elements; 44% reported eligibility criteria based on PICO, and 76% reported assessing the risk of bias of SRs), except for methods for summarizing evidence (20%) or statistical synthesis (26%). A minority reported methods for handling unique aspects of overviews (e.g., overlap in the primary studies [30%], discrepant or missing data [14%], and discordant results/conclusions across reviews [20%]). Conclusion: Reporting of methods unique to overviews requires improvement. Our findings provide a benchmark of the completeness of reporting and may inform guidance on the conduct and reporting of overviews.

AB - Objectives: The aim of the study was to assess the completeness of reporting of methods in overviews. Study Design and Setting: Assessment of the adequacy of reporting of methods in a random sample of 50 overviews was based on a published framework of methods for conducting overviews. Descriptive summary statistics were presented. Results: We screened 848 randomly selected abstracts to obtain the required 50 overviews. Overviews included a median of 13 (interquartile range 7–32) systematic reviews (SRs), 22% reported working from a protocol, 36% reported using reporting standards (e.g., Preferred Reporting Items for Systematic Reviews and Meta-Analyses), and 34% reported using methodological guidance (e.g., Cochrane Handbook). Methods common to both overviews and SRs of primary studies were reported in majority of overviews (e.g., 56% framed the overview question by Population, Intervention(s), Comparison(s), Outcome(s) [PICO] elements; 44% reported eligibility criteria based on PICO, and 76% reported assessing the risk of bias of SRs), except for methods for summarizing evidence (20%) or statistical synthesis (26%). A minority reported methods for handling unique aspects of overviews (e.g., overlap in the primary studies [30%], discrepant or missing data [14%], and discordant results/conclusions across reviews [20%]). Conclusion: Reporting of methods unique to overviews requires improvement. Our findings provide a benchmark of the completeness of reporting and may inform guidance on the conduct and reporting of overviews.

KW - Meta-review

KW - Overview methods

KW - Overviews of systematic reviews

KW - Reporting

KW - Systematic review methods

KW - Umbrella

UR - http://www.scopus.com/inward/record.url?scp=85076001101&partnerID=8YFLogxK

U2 - 10.1016/j.jclinepi.2019.09.025

DO - 10.1016/j.jclinepi.2019.09.025

M3 - Review Article

C2 - 31606430

VL - 118

SP - 69

EP - 85

JO - Journal of Clinical Epidemiology

JF - Journal of Clinical Epidemiology

SN - 0895-4356

ER -