Assessing the complexity of interventions within systematic reviews

Development, content and use of a new tool (iCAT-SR)

Simon Lewin, Maggie Hendry, Jackie Chandler, Andrew D Oxman, Susan Michie, Sasha Shepperd, Barnaby C. Reeves, Peter Tugwell, Karin Hannes, Eva A Rehfuess, Vivien Welch, Joanne E. McKenzie, Belinda Burford, Jennifer Petkovic, Laurie M Anderson, Janet Harris, Jane Noyes

Research output: Contribution to journalArticleResearchpeer-review

18 Citations (Scopus)

Abstract

Background: Health interventions fall along a spectrum from simple to more complex. There is wide interest in methods for reviewing 'complex interventions', but few transparent approaches for assessing intervention complexity in systematic reviews. Such assessments may assist review authors in, for example, systematically describing interventions and developing logic models. This paper describes the development and application of the intervention Complexity Assessment Tool for Systematic Reviews (iCAT-SR), a new tool to assess and categorise levels of intervention complexity in systematic reviews. Methods: We developed the iCAT-SR by adapting and extending an existing complexity assessment tool for randomized trials. We undertook this adaptation using a consensus approach in which possible complexity dimensions were circulated for feedback to a panel of methodologists with expertise in complex interventions and systematic reviews. Based on these inputs, we developed a draft version of the tool. We then invited a second round of feedback from the panel and a wider group of systematic reviewers. This informed further refinement of the tool. Results: The tool comprises ten dimensions: (1) the number of active components in the intervention; (2) the number of behaviours of recipients to which the intervention is directed; (3) the range and number of organizational levels targeted by the intervention; (4) the degree of tailoring intended or flexibility permitted across sites or individuals in applying or implementing the intervention; (5) the level of skill required by those delivering the intervention; (6) the level of skill required by those receiving the intervention; (7) the degree of interaction between intervention components; (8) the degree to which the effects of the intervention are context dependent; (9) the degree to which the effects of the interventions are changed by recipient or provider factors; (10) and the nature of the causal pathway between intervention and outcome. Dimensions 1-6 are considered 'core' dimensions. Dimensions 7-10 are optional and may not be useful for all interventions. Conclusions: The iCAT-SR tool facilitates more in-depth, systematic assessment of the complexity of interventions in systematic reviews and can assist in undertaking reviews and interpreting review findings. Further testing of the tool is now needed.

Original languageEnglish
Article number76
Number of pages13
JournalBMC Medical Research Methodology
Volume17
Issue number1
DOIs
Publication statusPublished - 26 Apr 2017

Keywords

  • Complex
  • Complex interventions
  • Complexity
  • Evidence synthesis
  • Intervention
  • Intervention development
  • Systematic review
  • Tool

Cite this

Lewin, Simon ; Hendry, Maggie ; Chandler, Jackie ; Oxman, Andrew D ; Michie, Susan ; Shepperd, Sasha ; Reeves, Barnaby C. ; Tugwell, Peter ; Hannes, Karin ; Rehfuess, Eva A ; Welch, Vivien ; McKenzie, Joanne E. ; Burford, Belinda ; Petkovic, Jennifer ; Anderson, Laurie M ; Harris, Janet ; Noyes, Jane. / Assessing the complexity of interventions within systematic reviews : Development, content and use of a new tool (iCAT-SR). In: BMC Medical Research Methodology. 2017 ; Vol. 17, No. 1.
@article{7b9853dacae54269be20e5bb1ab1d553,
title = "Assessing the complexity of interventions within systematic reviews: Development, content and use of a new tool (iCAT-SR)",
abstract = "Background: Health interventions fall along a spectrum from simple to more complex. There is wide interest in methods for reviewing 'complex interventions', but few transparent approaches for assessing intervention complexity in systematic reviews. Such assessments may assist review authors in, for example, systematically describing interventions and developing logic models. This paper describes the development and application of the intervention Complexity Assessment Tool for Systematic Reviews (iCAT-SR), a new tool to assess and categorise levels of intervention complexity in systematic reviews. Methods: We developed the iCAT-SR by adapting and extending an existing complexity assessment tool for randomized trials. We undertook this adaptation using a consensus approach in which possible complexity dimensions were circulated for feedback to a panel of methodologists with expertise in complex interventions and systematic reviews. Based on these inputs, we developed a draft version of the tool. We then invited a second round of feedback from the panel and a wider group of systematic reviewers. This informed further refinement of the tool. Results: The tool comprises ten dimensions: (1) the number of active components in the intervention; (2) the number of behaviours of recipients to which the intervention is directed; (3) the range and number of organizational levels targeted by the intervention; (4) the degree of tailoring intended or flexibility permitted across sites or individuals in applying or implementing the intervention; (5) the level of skill required by those delivering the intervention; (6) the level of skill required by those receiving the intervention; (7) the degree of interaction between intervention components; (8) the degree to which the effects of the intervention are context dependent; (9) the degree to which the effects of the interventions are changed by recipient or provider factors; (10) and the nature of the causal pathway between intervention and outcome. Dimensions 1-6 are considered 'core' dimensions. Dimensions 7-10 are optional and may not be useful for all interventions. Conclusions: The iCAT-SR tool facilitates more in-depth, systematic assessment of the complexity of interventions in systematic reviews and can assist in undertaking reviews and interpreting review findings. Further testing of the tool is now needed.",
keywords = "Complex, Complex interventions, Complexity, Evidence synthesis, Intervention, Intervention development, Systematic review, Tool",
author = "Simon Lewin and Maggie Hendry and Jackie Chandler and Oxman, {Andrew D} and Susan Michie and Sasha Shepperd and Reeves, {Barnaby C.} and Peter Tugwell and Karin Hannes and Rehfuess, {Eva A} and Vivien Welch and McKenzie, {Joanne E.} and Belinda Burford and Jennifer Petkovic and Anderson, {Laurie M} and Janet Harris and Jane Noyes",
year = "2017",
month = "4",
day = "26",
doi = "10.1186/s12874-017-0349-x",
language = "English",
volume = "17",
journal = "BMC Medical Research Methodology",
issn = "1471-2288",
publisher = "Springer-Verlag London Ltd.",
number = "1",

}

Lewin, S, Hendry, M, Chandler, J, Oxman, AD, Michie, S, Shepperd, S, Reeves, BC, Tugwell, P, Hannes, K, Rehfuess, EA, Welch, V, McKenzie, JE, Burford, B, Petkovic, J, Anderson, LM, Harris, J & Noyes, J 2017, 'Assessing the complexity of interventions within systematic reviews: Development, content and use of a new tool (iCAT-SR)', BMC Medical Research Methodology, vol. 17, no. 1, 76. https://doi.org/10.1186/s12874-017-0349-x

Assessing the complexity of interventions within systematic reviews : Development, content and use of a new tool (iCAT-SR). / Lewin, Simon; Hendry, Maggie; Chandler, Jackie; Oxman, Andrew D; Michie, Susan; Shepperd, Sasha; Reeves, Barnaby C.; Tugwell, Peter; Hannes, Karin; Rehfuess, Eva A; Welch, Vivien; McKenzie, Joanne E.; Burford, Belinda; Petkovic, Jennifer; Anderson, Laurie M; Harris, Janet; Noyes, Jane.

In: BMC Medical Research Methodology, Vol. 17, No. 1, 76, 26.04.2017.

Research output: Contribution to journalArticleResearchpeer-review

TY - JOUR

T1 - Assessing the complexity of interventions within systematic reviews

T2 - Development, content and use of a new tool (iCAT-SR)

AU - Lewin, Simon

AU - Hendry, Maggie

AU - Chandler, Jackie

AU - Oxman, Andrew D

AU - Michie, Susan

AU - Shepperd, Sasha

AU - Reeves, Barnaby C.

AU - Tugwell, Peter

AU - Hannes, Karin

AU - Rehfuess, Eva A

AU - Welch, Vivien

AU - McKenzie, Joanne E.

AU - Burford, Belinda

AU - Petkovic, Jennifer

AU - Anderson, Laurie M

AU - Harris, Janet

AU - Noyes, Jane

PY - 2017/4/26

Y1 - 2017/4/26

N2 - Background: Health interventions fall along a spectrum from simple to more complex. There is wide interest in methods for reviewing 'complex interventions', but few transparent approaches for assessing intervention complexity in systematic reviews. Such assessments may assist review authors in, for example, systematically describing interventions and developing logic models. This paper describes the development and application of the intervention Complexity Assessment Tool for Systematic Reviews (iCAT-SR), a new tool to assess and categorise levels of intervention complexity in systematic reviews. Methods: We developed the iCAT-SR by adapting and extending an existing complexity assessment tool for randomized trials. We undertook this adaptation using a consensus approach in which possible complexity dimensions were circulated for feedback to a panel of methodologists with expertise in complex interventions and systematic reviews. Based on these inputs, we developed a draft version of the tool. We then invited a second round of feedback from the panel and a wider group of systematic reviewers. This informed further refinement of the tool. Results: The tool comprises ten dimensions: (1) the number of active components in the intervention; (2) the number of behaviours of recipients to which the intervention is directed; (3) the range and number of organizational levels targeted by the intervention; (4) the degree of tailoring intended or flexibility permitted across sites or individuals in applying or implementing the intervention; (5) the level of skill required by those delivering the intervention; (6) the level of skill required by those receiving the intervention; (7) the degree of interaction between intervention components; (8) the degree to which the effects of the intervention are context dependent; (9) the degree to which the effects of the interventions are changed by recipient or provider factors; (10) and the nature of the causal pathway between intervention and outcome. Dimensions 1-6 are considered 'core' dimensions. Dimensions 7-10 are optional and may not be useful for all interventions. Conclusions: The iCAT-SR tool facilitates more in-depth, systematic assessment of the complexity of interventions in systematic reviews and can assist in undertaking reviews and interpreting review findings. Further testing of the tool is now needed.

AB - Background: Health interventions fall along a spectrum from simple to more complex. There is wide interest in methods for reviewing 'complex interventions', but few transparent approaches for assessing intervention complexity in systematic reviews. Such assessments may assist review authors in, for example, systematically describing interventions and developing logic models. This paper describes the development and application of the intervention Complexity Assessment Tool for Systematic Reviews (iCAT-SR), a new tool to assess and categorise levels of intervention complexity in systematic reviews. Methods: We developed the iCAT-SR by adapting and extending an existing complexity assessment tool for randomized trials. We undertook this adaptation using a consensus approach in which possible complexity dimensions were circulated for feedback to a panel of methodologists with expertise in complex interventions and systematic reviews. Based on these inputs, we developed a draft version of the tool. We then invited a second round of feedback from the panel and a wider group of systematic reviewers. This informed further refinement of the tool. Results: The tool comprises ten dimensions: (1) the number of active components in the intervention; (2) the number of behaviours of recipients to which the intervention is directed; (3) the range and number of organizational levels targeted by the intervention; (4) the degree of tailoring intended or flexibility permitted across sites or individuals in applying or implementing the intervention; (5) the level of skill required by those delivering the intervention; (6) the level of skill required by those receiving the intervention; (7) the degree of interaction between intervention components; (8) the degree to which the effects of the intervention are context dependent; (9) the degree to which the effects of the interventions are changed by recipient or provider factors; (10) and the nature of the causal pathway between intervention and outcome. Dimensions 1-6 are considered 'core' dimensions. Dimensions 7-10 are optional and may not be useful for all interventions. Conclusions: The iCAT-SR tool facilitates more in-depth, systematic assessment of the complexity of interventions in systematic reviews and can assist in undertaking reviews and interpreting review findings. Further testing of the tool is now needed.

KW - Complex

KW - Complex interventions

KW - Complexity

KW - Evidence synthesis

KW - Intervention

KW - Intervention development

KW - Systematic review

KW - Tool

UR - http://www.scopus.com/inward/record.url?scp=85018837586&partnerID=8YFLogxK

U2 - 10.1186/s12874-017-0349-x

DO - 10.1186/s12874-017-0349-x

M3 - Article

VL - 17

JO - BMC Medical Research Methodology

JF - BMC Medical Research Methodology

SN - 1471-2288

IS - 1

M1 - 76

ER -