Reach, engagement, and effectiveness: A systematic review of evaluation methodologies used in health promotion via social networking sites

Research output: Contribution to journalArticleResearchpeer-review

Abstract

Issue addressed Social networking sites (SNS) are increasingly popular platforms for health promotion. Advancements in SNS health promotion require quality evidence; however, interventions are often not formally evaluated. This study aims to describe evaluation practices used in SNS health promotion. Methods A systematic review was undertaken of Medline, PsycINFO, Scopus, EMBASE, CINAHL Plus, Communication and Mass Media Complete, and Cochrane Library databases. Articles published between 2006 and 2013 describing any health promotion intervention delivered using SNS were included. Results Forty-seven studies were included. There were two main evaluation approaches: closed designs (n≤23), which used traditional research designs and formal recruitment procedures; and open designs (n≤19), which evaluated the intervention in a real-world setting, allowing unknown SNS users to interact with the content without enrolling in research. Closed designs were unable to assess reach and engagement beyond their research sample. Open designs often relied on weaker study designs with no use of objective outcome measures and yielded low response rates. Conclusions Barriers to evaluation included low participation rates, high attrition, unknown representativeness and lack of comparison groups. Acceptability was typically assessed among those engaged with the intervention, with limited population data available to accurately assess intervention reach. Few studies were able to assess uptake of the intervention in a real-life setting while simultaneously assessing effectiveness of interventions with research rigour. So what? Through use of quasi-experimental or well designed before-after evaluations, in combination with detailed engagement metrics, it is possible to balance assessment of effectiveness and reach to evaluate SNS health promotion.

Original languageEnglish
Pages (from-to)187-197
Number of pages11
JournalHealth Promotion Journal of Australia
Volume27
Issue number3
DOIs
Publication statusPublished - 1 Dec 2016

Keywords

  • behaviour change
  • evaluation methods
  • information and communication technology

Cite this

@article{fae6ef60c8e64df4af71c64495599ec4,
title = "Reach, engagement, and effectiveness: A systematic review of evaluation methodologies used in health promotion via social networking sites",
abstract = "Issue addressed Social networking sites (SNS) are increasingly popular platforms for health promotion. Advancements in SNS health promotion require quality evidence; however, interventions are often not formally evaluated. This study aims to describe evaluation practices used in SNS health promotion. Methods A systematic review was undertaken of Medline, PsycINFO, Scopus, EMBASE, CINAHL Plus, Communication and Mass Media Complete, and Cochrane Library databases. Articles published between 2006 and 2013 describing any health promotion intervention delivered using SNS were included. Results Forty-seven studies were included. There were two main evaluation approaches: closed designs (n≤23), which used traditional research designs and formal recruitment procedures; and open designs (n≤19), which evaluated the intervention in a real-world setting, allowing unknown SNS users to interact with the content without enrolling in research. Closed designs were unable to assess reach and engagement beyond their research sample. Open designs often relied on weaker study designs with no use of objective outcome measures and yielded low response rates. Conclusions Barriers to evaluation included low participation rates, high attrition, unknown representativeness and lack of comparison groups. Acceptability was typically assessed among those engaged with the intervention, with limited population data available to accurately assess intervention reach. Few studies were able to assess uptake of the intervention in a real-life setting while simultaneously assessing effectiveness of interventions with research rigour. So what? Through use of quasi-experimental or well designed before-after evaluations, in combination with detailed engagement metrics, it is possible to balance assessment of effectiveness and reach to evaluate SNS health promotion.",
keywords = "behaviour change, evaluation methods, information and communication technology",
author = "Lim, {Megan S.C.} and Wright, {Cassandra J.C.} and Carrotte, {Elise R.} and Pedrana, {Alisa E.}",
year = "2016",
month = "12",
day = "1",
doi = "10.1071/HE16057",
language = "English",
volume = "27",
pages = "187--197",
journal = "Health Promotion Journal of Australia",
issn = "1036-1073",
publisher = "John Wiley & Sons",
number = "3",

}

TY - JOUR

T1 - Reach, engagement, and effectiveness

T2 - A systematic review of evaluation methodologies used in health promotion via social networking sites

AU - Lim, Megan S.C.

AU - Wright, Cassandra J.C.

AU - Carrotte, Elise R.

AU - Pedrana, Alisa E.

PY - 2016/12/1

Y1 - 2016/12/1

N2 - Issue addressed Social networking sites (SNS) are increasingly popular platforms for health promotion. Advancements in SNS health promotion require quality evidence; however, interventions are often not formally evaluated. This study aims to describe evaluation practices used in SNS health promotion. Methods A systematic review was undertaken of Medline, PsycINFO, Scopus, EMBASE, CINAHL Plus, Communication and Mass Media Complete, and Cochrane Library databases. Articles published between 2006 and 2013 describing any health promotion intervention delivered using SNS were included. Results Forty-seven studies were included. There were two main evaluation approaches: closed designs (n≤23), which used traditional research designs and formal recruitment procedures; and open designs (n≤19), which evaluated the intervention in a real-world setting, allowing unknown SNS users to interact with the content without enrolling in research. Closed designs were unable to assess reach and engagement beyond their research sample. Open designs often relied on weaker study designs with no use of objective outcome measures and yielded low response rates. Conclusions Barriers to evaluation included low participation rates, high attrition, unknown representativeness and lack of comparison groups. Acceptability was typically assessed among those engaged with the intervention, with limited population data available to accurately assess intervention reach. Few studies were able to assess uptake of the intervention in a real-life setting while simultaneously assessing effectiveness of interventions with research rigour. So what? Through use of quasi-experimental or well designed before-after evaluations, in combination with detailed engagement metrics, it is possible to balance assessment of effectiveness and reach to evaluate SNS health promotion.

AB - Issue addressed Social networking sites (SNS) are increasingly popular platforms for health promotion. Advancements in SNS health promotion require quality evidence; however, interventions are often not formally evaluated. This study aims to describe evaluation practices used in SNS health promotion. Methods A systematic review was undertaken of Medline, PsycINFO, Scopus, EMBASE, CINAHL Plus, Communication and Mass Media Complete, and Cochrane Library databases. Articles published between 2006 and 2013 describing any health promotion intervention delivered using SNS were included. Results Forty-seven studies were included. There were two main evaluation approaches: closed designs (n≤23), which used traditional research designs and formal recruitment procedures; and open designs (n≤19), which evaluated the intervention in a real-world setting, allowing unknown SNS users to interact with the content without enrolling in research. Closed designs were unable to assess reach and engagement beyond their research sample. Open designs often relied on weaker study designs with no use of objective outcome measures and yielded low response rates. Conclusions Barriers to evaluation included low participation rates, high attrition, unknown representativeness and lack of comparison groups. Acceptability was typically assessed among those engaged with the intervention, with limited population data available to accurately assess intervention reach. Few studies were able to assess uptake of the intervention in a real-life setting while simultaneously assessing effectiveness of interventions with research rigour. So what? Through use of quasi-experimental or well designed before-after evaluations, in combination with detailed engagement metrics, it is possible to balance assessment of effectiveness and reach to evaluate SNS health promotion.

KW - behaviour change

KW - evaluation methods

KW - information and communication technology

UR - http://www.scopus.com/inward/record.url?scp=85002673995&partnerID=8YFLogxK

U2 - 10.1071/HE16057

DO - 10.1071/HE16057

M3 - Article

VL - 27

SP - 187

EP - 197

JO - Health Promotion Journal of Australia

JF - Health Promotion Journal of Australia

SN - 1036-1073

IS - 3

ER -