Same family, different species

Methodological conduct and quality varies according to purpose for five types of knowledge synthesis

Andrea C. Tricco, Wasifa Zarin, Marco Ghassemi, Vera Nincic, Erin Lillie, Matthew J. Page, Larissa Shamseer, Jesmin Antony, Patricia Rios, Jeremiah Hwee, Areti Angeliki Veroniki, David Moher, Lisa Hartling, Ba' Pham, Sharon E. Straus

Research output: Contribution to journalArticleResearchpeer-review

10 Citations (Scopus)

Abstract

Objectives: The aim of the study was to characterize methodological conduct, reporting, and quality of five knowledge synthesis (KS) approaches. Study Design and Setting: Retrospective analysis of a convenience sample of five published databases of KS approaches: overview of reviews (n = 74), scoping reviews (n = 494), rapid reviews (n = 84), systematic reviews (n = 300), and network meta-analyses (NMAs; n = 456). Data in the five published databases were abstracted by two reviewers independently, any missing data for this retrospective analysis were abstracted by one experienced reviewer. Methods were appraised using the A MeaSurement Tool to Assess systematic Reviews (AMSTAR) tool. Descriptive analysis was performed. Results: Reporting the use of a protocol ranged from 4% for rapid reviews to 32% for systematic reviews. The use of two reviewers for citation and full-text screening ranged from 20% for scoping reviews to 60% for NMAs. Data abstraction was performed in duplicate for 11% of rapid reviews and 54% of NMAs, and for risk of bias appraisal, this ranged from 6% for scoping reviews to 41% for NMAs. NMAs had the highest median percentage of maximum obtainable AMSTAR score (64%; Q1-Q3:45-73%), while scoping reviews had the lowest (25%; Q1-Q3:13-38%). Conclusion: NMAs consistently scored the highest on the AMSTAR tool likely because the purpose is to estimate treatment effects statistically. Scoping reviews scored the lowest (even after adjusting the score for not relevant items) likely because the purpose is to characterize the literature.

Original languageEnglish
Pages (from-to)133-142
Number of pages10
JournalJournal of Clinical Epidemiology
Volume96
DOIs
Publication statusPublished - Apr 2017

Keywords

  • Knowledge synthesis
  • Network meta-analysis
  • Overview of reviews
  • Rapid review
  • Scoping review
  • Systematic review

Cite this

Tricco, Andrea C. ; Zarin, Wasifa ; Ghassemi, Marco ; Nincic, Vera ; Lillie, Erin ; Page, Matthew J. ; Shamseer, Larissa ; Antony, Jesmin ; Rios, Patricia ; Hwee, Jeremiah ; Veroniki, Areti Angeliki ; Moher, David ; Hartling, Lisa ; Pham, Ba' ; Straus, Sharon E. / Same family, different species : Methodological conduct and quality varies according to purpose for five types of knowledge synthesis. In: Journal of Clinical Epidemiology. 2017 ; Vol. 96. pp. 133-142.
@article{843e79e47256428e83f6407383b6d296,
title = "Same family, different species: Methodological conduct and quality varies according to purpose for five types of knowledge synthesis",
abstract = "Objectives: The aim of the study was to characterize methodological conduct, reporting, and quality of five knowledge synthesis (KS) approaches. Study Design and Setting: Retrospective analysis of a convenience sample of five published databases of KS approaches: overview of reviews (n = 74), scoping reviews (n = 494), rapid reviews (n = 84), systematic reviews (n = 300), and network meta-analyses (NMAs; n = 456). Data in the five published databases were abstracted by two reviewers independently, any missing data for this retrospective analysis were abstracted by one experienced reviewer. Methods were appraised using the A MeaSurement Tool to Assess systematic Reviews (AMSTAR) tool. Descriptive analysis was performed. Results: Reporting the use of a protocol ranged from 4{\%} for rapid reviews to 32{\%} for systematic reviews. The use of two reviewers for citation and full-text screening ranged from 20{\%} for scoping reviews to 60{\%} for NMAs. Data abstraction was performed in duplicate for 11{\%} of rapid reviews and 54{\%} of NMAs, and for risk of bias appraisal, this ranged from 6{\%} for scoping reviews to 41{\%} for NMAs. NMAs had the highest median percentage of maximum obtainable AMSTAR score (64{\%}; Q1-Q3:45-73{\%}), while scoping reviews had the lowest (25{\%}; Q1-Q3:13-38{\%}). Conclusion: NMAs consistently scored the highest on the AMSTAR tool likely because the purpose is to estimate treatment effects statistically. Scoping reviews scored the lowest (even after adjusting the score for not relevant items) likely because the purpose is to characterize the literature.",
keywords = "Knowledge synthesis, Network meta-analysis, Overview of reviews, Rapid review, Scoping review, Systematic review",
author = "Tricco, {Andrea C.} and Wasifa Zarin and Marco Ghassemi and Vera Nincic and Erin Lillie and Page, {Matthew J.} and Larissa Shamseer and Jesmin Antony and Patricia Rios and Jeremiah Hwee and Veroniki, {Areti Angeliki} and David Moher and Lisa Hartling and Ba' Pham and Straus, {Sharon E.}",
year = "2017",
month = "4",
doi = "10.1016/j.jclinepi.2017.10.014",
language = "English",
volume = "96",
pages = "133--142",
journal = "Journal of Clinical Epidemiology",
issn = "0895-4356",
publisher = "Elsevier",

}

Tricco, AC, Zarin, W, Ghassemi, M, Nincic, V, Lillie, E, Page, MJ, Shamseer, L, Antony, J, Rios, P, Hwee, J, Veroniki, AA, Moher, D, Hartling, L, Pham, B & Straus, SE 2017, 'Same family, different species: Methodological conduct and quality varies according to purpose for five types of knowledge synthesis', Journal of Clinical Epidemiology, vol. 96, pp. 133-142. https://doi.org/10.1016/j.jclinepi.2017.10.014

Same family, different species : Methodological conduct and quality varies according to purpose for five types of knowledge synthesis. / Tricco, Andrea C.; Zarin, Wasifa; Ghassemi, Marco; Nincic, Vera; Lillie, Erin; Page, Matthew J.; Shamseer, Larissa; Antony, Jesmin; Rios, Patricia; Hwee, Jeremiah; Veroniki, Areti Angeliki; Moher, David; Hartling, Lisa; Pham, Ba'; Straus, Sharon E.

In: Journal of Clinical Epidemiology, Vol. 96, 04.2017, p. 133-142.

Research output: Contribution to journalArticleResearchpeer-review

TY - JOUR

T1 - Same family, different species

T2 - Methodological conduct and quality varies according to purpose for five types of knowledge synthesis

AU - Tricco, Andrea C.

AU - Zarin, Wasifa

AU - Ghassemi, Marco

AU - Nincic, Vera

AU - Lillie, Erin

AU - Page, Matthew J.

AU - Shamseer, Larissa

AU - Antony, Jesmin

AU - Rios, Patricia

AU - Hwee, Jeremiah

AU - Veroniki, Areti Angeliki

AU - Moher, David

AU - Hartling, Lisa

AU - Pham, Ba'

AU - Straus, Sharon E.

PY - 2017/4

Y1 - 2017/4

N2 - Objectives: The aim of the study was to characterize methodological conduct, reporting, and quality of five knowledge synthesis (KS) approaches. Study Design and Setting: Retrospective analysis of a convenience sample of five published databases of KS approaches: overview of reviews (n = 74), scoping reviews (n = 494), rapid reviews (n = 84), systematic reviews (n = 300), and network meta-analyses (NMAs; n = 456). Data in the five published databases were abstracted by two reviewers independently, any missing data for this retrospective analysis were abstracted by one experienced reviewer. Methods were appraised using the A MeaSurement Tool to Assess systematic Reviews (AMSTAR) tool. Descriptive analysis was performed. Results: Reporting the use of a protocol ranged from 4% for rapid reviews to 32% for systematic reviews. The use of two reviewers for citation and full-text screening ranged from 20% for scoping reviews to 60% for NMAs. Data abstraction was performed in duplicate for 11% of rapid reviews and 54% of NMAs, and for risk of bias appraisal, this ranged from 6% for scoping reviews to 41% for NMAs. NMAs had the highest median percentage of maximum obtainable AMSTAR score (64%; Q1-Q3:45-73%), while scoping reviews had the lowest (25%; Q1-Q3:13-38%). Conclusion: NMAs consistently scored the highest on the AMSTAR tool likely because the purpose is to estimate treatment effects statistically. Scoping reviews scored the lowest (even after adjusting the score for not relevant items) likely because the purpose is to characterize the literature.

AB - Objectives: The aim of the study was to characterize methodological conduct, reporting, and quality of five knowledge synthesis (KS) approaches. Study Design and Setting: Retrospective analysis of a convenience sample of five published databases of KS approaches: overview of reviews (n = 74), scoping reviews (n = 494), rapid reviews (n = 84), systematic reviews (n = 300), and network meta-analyses (NMAs; n = 456). Data in the five published databases were abstracted by two reviewers independently, any missing data for this retrospective analysis were abstracted by one experienced reviewer. Methods were appraised using the A MeaSurement Tool to Assess systematic Reviews (AMSTAR) tool. Descriptive analysis was performed. Results: Reporting the use of a protocol ranged from 4% for rapid reviews to 32% for systematic reviews. The use of two reviewers for citation and full-text screening ranged from 20% for scoping reviews to 60% for NMAs. Data abstraction was performed in duplicate for 11% of rapid reviews and 54% of NMAs, and for risk of bias appraisal, this ranged from 6% for scoping reviews to 41% for NMAs. NMAs had the highest median percentage of maximum obtainable AMSTAR score (64%; Q1-Q3:45-73%), while scoping reviews had the lowest (25%; Q1-Q3:13-38%). Conclusion: NMAs consistently scored the highest on the AMSTAR tool likely because the purpose is to estimate treatment effects statistically. Scoping reviews scored the lowest (even after adjusting the score for not relevant items) likely because the purpose is to characterize the literature.

KW - Knowledge synthesis

KW - Network meta-analysis

KW - Overview of reviews

KW - Rapid review

KW - Scoping review

KW - Systematic review

UR - http://www.scopus.com/inward/record.url?scp=85034953309&partnerID=8YFLogxK

U2 - 10.1016/j.jclinepi.2017.10.014

DO - 10.1016/j.jclinepi.2017.10.014

M3 - Article

VL - 96

SP - 133

EP - 142

JO - Journal of Clinical Epidemiology

JF - Journal of Clinical Epidemiology

SN - 0895-4356

ER -