Improving the conduct of systematic reviews

a process mining perspective

Ba Pham, Ebrahim Bagheri, Patricia Rios, Asef Pourmasoumi, Reid Robson, Jeremiah Hwee, Wanrudee Isaranuwatchai, Nazia Darvesh, Matthew James Page, Andrea C. Tricco

Research output: Contribution to journalReview ArticleResearchpeer-review

1 Citation (Scopus)

Abstract

Objectives
To illustrate the use of process mining concepts, techniques, and tools to improve the systematic review process.

Study Design and Setting
We simulated review activities and step-specific methods in the process for systematic reviews conducted by one research team over 1 year to generate an event log of activities, with start/end dates, reviewer assignment by expertise, and person-hours worked. Process mining techniques were applied to the event log to “discover” process models, which allowed visual display, animation, or replay of the simulated review activities. Summary statistics were calculated for person-time and timelines. We also analyzed the social networks of team interactions.

Results
The 12 simulated reviews included an average of 3,831 titles/abstracts (range: 1,565–6,368) and 20 studies (6–42). The average review completion time was 463 days (range: 289–629) (881 person-hours [range: 243–1,752]). The average person-hours per activity were study selection 26%, data collection 24%, report preparation 23%, and meta-analysis 17%. Social network analyses showed the organizational interaction of team members, including how they worked together to complete review tasks and to hand over tasks upon completion.

Conclusion
Event log and process mining can be valuable tools for research teams interested in improving and modernizing the systematic review process.
Original languageEnglish
Pages (from-to)101-111
Number of pages11
JournalJournal of Clinical Epidemiology
Volume103
DOIs
Publication statusPublished - Nov 2018

Cite this

Pham, B., Bagheri, E., Rios, P., Pourmasoumi, A., Robson, R., Hwee, J., ... Tricco, A. C. (2018). Improving the conduct of systematic reviews: a process mining perspective. Journal of Clinical Epidemiology, 103, 101-111. https://doi.org/10.1016/j.jclinepi.2018.06.011
Pham, Ba ; Bagheri, Ebrahim ; Rios, Patricia ; Pourmasoumi, Asef ; Robson, Reid ; Hwee, Jeremiah ; Isaranuwatchai, Wanrudee ; Darvesh, Nazia ; Page, Matthew James ; Tricco, Andrea C. / Improving the conduct of systematic reviews : a process mining perspective. In: Journal of Clinical Epidemiology. 2018 ; Vol. 103. pp. 101-111.
@article{9a881668fbb54b3fb255b3358f9a38fb,
title = "Improving the conduct of systematic reviews: a process mining perspective",
abstract = "ObjectivesTo illustrate the use of process mining concepts, techniques, and tools to improve the systematic review process.Study Design and SettingWe simulated review activities and step-specific methods in the process for systematic reviews conducted by one research team over 1 year to generate an event log of activities, with start/end dates, reviewer assignment by expertise, and person-hours worked. Process mining techniques were applied to the event log to “discover” process models, which allowed visual display, animation, or replay of the simulated review activities. Summary statistics were calculated for person-time and timelines. We also analyzed the social networks of team interactions.ResultsThe 12 simulated reviews included an average of 3,831 titles/abstracts (range: 1,565–6,368) and 20 studies (6–42). The average review completion time was 463 days (range: 289–629) (881 person-hours [range: 243–1,752]). The average person-hours per activity were study selection 26{\%}, data collection 24{\%}, report preparation 23{\%}, and meta-analysis 17{\%}. Social network analyses showed the organizational interaction of team members, including how they worked together to complete review tasks and to hand over tasks upon completion.ConclusionEvent log and process mining can be valuable tools for research teams interested in improving and modernizing the systematic review process.",
author = "Ba Pham and Ebrahim Bagheri and Patricia Rios and Asef Pourmasoumi and Reid Robson and Jeremiah Hwee and Wanrudee Isaranuwatchai and Nazia Darvesh and Page, {Matthew James} and Tricco, {Andrea C.}",
year = "2018",
month = "11",
doi = "10.1016/j.jclinepi.2018.06.011",
language = "English",
volume = "103",
pages = "101--111",
journal = "Journal of Clinical Epidemiology",
issn = "0895-4356",
publisher = "Elsevier",

}

Pham, B, Bagheri, E, Rios, P, Pourmasoumi, A, Robson, R, Hwee, J, Isaranuwatchai, W, Darvesh, N, Page, MJ & Tricco, AC 2018, 'Improving the conduct of systematic reviews: a process mining perspective', Journal of Clinical Epidemiology, vol. 103, pp. 101-111. https://doi.org/10.1016/j.jclinepi.2018.06.011

Improving the conduct of systematic reviews : a process mining perspective. / Pham, Ba; Bagheri, Ebrahim; Rios, Patricia; Pourmasoumi, Asef; Robson, Reid; Hwee, Jeremiah; Isaranuwatchai, Wanrudee; Darvesh, Nazia; Page, Matthew James; Tricco, Andrea C.

In: Journal of Clinical Epidemiology, Vol. 103, 11.2018, p. 101-111.

Research output: Contribution to journalReview ArticleResearchpeer-review

TY - JOUR

T1 - Improving the conduct of systematic reviews

T2 - a process mining perspective

AU - Pham, Ba

AU - Bagheri, Ebrahim

AU - Rios, Patricia

AU - Pourmasoumi, Asef

AU - Robson, Reid

AU - Hwee, Jeremiah

AU - Isaranuwatchai, Wanrudee

AU - Darvesh, Nazia

AU - Page, Matthew James

AU - Tricco, Andrea C.

PY - 2018/11

Y1 - 2018/11

N2 - ObjectivesTo illustrate the use of process mining concepts, techniques, and tools to improve the systematic review process.Study Design and SettingWe simulated review activities and step-specific methods in the process for systematic reviews conducted by one research team over 1 year to generate an event log of activities, with start/end dates, reviewer assignment by expertise, and person-hours worked. Process mining techniques were applied to the event log to “discover” process models, which allowed visual display, animation, or replay of the simulated review activities. Summary statistics were calculated for person-time and timelines. We also analyzed the social networks of team interactions.ResultsThe 12 simulated reviews included an average of 3,831 titles/abstracts (range: 1,565–6,368) and 20 studies (6–42). The average review completion time was 463 days (range: 289–629) (881 person-hours [range: 243–1,752]). The average person-hours per activity were study selection 26%, data collection 24%, report preparation 23%, and meta-analysis 17%. Social network analyses showed the organizational interaction of team members, including how they worked together to complete review tasks and to hand over tasks upon completion.ConclusionEvent log and process mining can be valuable tools for research teams interested in improving and modernizing the systematic review process.

AB - ObjectivesTo illustrate the use of process mining concepts, techniques, and tools to improve the systematic review process.Study Design and SettingWe simulated review activities and step-specific methods in the process for systematic reviews conducted by one research team over 1 year to generate an event log of activities, with start/end dates, reviewer assignment by expertise, and person-hours worked. Process mining techniques were applied to the event log to “discover” process models, which allowed visual display, animation, or replay of the simulated review activities. Summary statistics were calculated for person-time and timelines. We also analyzed the social networks of team interactions.ResultsThe 12 simulated reviews included an average of 3,831 titles/abstracts (range: 1,565–6,368) and 20 studies (6–42). The average review completion time was 463 days (range: 289–629) (881 person-hours [range: 243–1,752]). The average person-hours per activity were study selection 26%, data collection 24%, report preparation 23%, and meta-analysis 17%. Social network analyses showed the organizational interaction of team members, including how they worked together to complete review tasks and to hand over tasks upon completion.ConclusionEvent log and process mining can be valuable tools for research teams interested in improving and modernizing the systematic review process.

U2 - 10.1016/j.jclinepi.2018.06.011

DO - 10.1016/j.jclinepi.2018.06.011

M3 - Review Article

VL - 103

SP - 101

EP - 111

JO - Journal of Clinical Epidemiology

JF - Journal of Clinical Epidemiology

SN - 0895-4356

ER -