Improving the “science” in evaluation of implementation science strategies

Type of session: Panel 19

Theme: 4. Innovation in implementation science

Objectives/aims
The aim of this panel will be to spark debate and discuss recent findings related to enhanced methodologies for the evaluation of implementation interventions in three key areas;
1) Research design
2) Outcome measurement selection
3) Consumer engagement
This panel has been prompted by work of the Evidence Translation in Allied Health (EviTAH) project, a 5-year NHMRC Partnership Project that has sought to identify more effective and cost-effective means for translating evidence into practice amongst allied health professionals working in health services.
Although our work has been focused in the context of allied health, this panel will focus on the underlying concepts and principles of evaluation in implementation science so that it is accessible to a broad audience.

Methods
Our panel will have three sections. Each will highlight current issues, draw examples from our research, and be followed by audience Q&A / discussion. Panel members are our presenting authors.

Section 1) Using experimental designs for knowledge translation strategies.
We draw from two innovative randomised controlled trial designs.

Section 2) Outcome measurement that reflects intended change.
We debate process vs health outcome measurement selection and highlight a new scale of organisational decision-making process.

Section 3) Measuring the impact of consumer engagement.
We highlight strengths and unanticipated limitations and draw implications for how consumer engagement should be quantified.


Main findings
Our first randomised trial involved n=45 health services and found that both guideline provision and provision of a brief knowledge broker intervention were ineffective for changing practice. Our second involved n=119 health professionals at one health service. We found that the provision of video-based research abstracts enhanced health professional understanding of research findings across three context areas compared to written abstracts and no intervention control.
We expect this panel to raise the profile and use of experimental methods for evaluating the effectiveness of interventions aiming to enhance use of evidence in practice.