Approaches to distributed execution of scientific workflows in Kepler

Marcin Płóciennik; Tomasz Zok; Ilkay Altintas; Jianwu Wang; Daniel Crawl; David Abramson; Frederic Imbeaux; Bernard Guillerminet; Marcos Lopez-Caniego; Isabel Campos Plasencia; Wojciech Pych; Pawel Cieciela̧g; Bartek Palak; Michał Owsiak; Yann Frauel

doi:10.3233/FI-2013-947

Approaches to distributed execution of scientific workflows in Kepler

Marcin Płóciennik, Tomasz Zok, Ilkay Altintas, Jianwu Wang, Daniel Crawl, David Abramson, Frederic Imbeaux, Bernard Guillerminet, Marcos Lopez-Caniego, Isabel Campos Plasencia, Wojciech Pych, Pawel Cieciela̧g, Bartek Palak, Michał Owsiak, Yann Frauel

Research output: Contribution to journal › Article › Research › peer-review

16 Citations (Scopus)

Abstract

The Kepler scientific workflow system enables creation, execution and sharing of workflows across a broad range of scientific and engineering disciplines while also facilitating remote and distributed execution of workflows. In this paper, we present and compare different approaches to distributed execution of workflows using the Kepler environment, including a distributed data-parallel framework using Hadoop and Stratosphere, and Cloud and Grid execution using Serpens, Nimrod/K and Globus actors. We also present real-life applications in computational chemistry, bioinformatics and computational physics to demonstrate the usage of different distributed computing capabilities of Kepler in executable workflows. We further analyze the differences of each approach and provide a guidance for their applications.

Original language	English
Pages (from-to)	281-302
Number of pages	22
Journal	Fundamenta Informaticae
Volume	128
Issue number	3
DOIs	https://doi.org/10.3233/FI-2013-947
Publication status	Published - 2013

Keywords

distributed execution
Kepler
scientific workflow

Access to Document

10.3233/FI-2013-947

Cite this

@article{810d0298ec5e4ffeb0681b2c54670f00,

title = "Approaches to distributed execution of scientific workflows in Kepler",

abstract = "The Kepler scientific workflow system enables creation, execution and sharing of workflows across a broad range of scientific and engineering disciplines while also facilitating remote and distributed execution of workflows. In this paper, we present and compare different approaches to distributed execution of workflows using the Kepler environment, including a distributed data-parallel framework using Hadoop and Stratosphere, and Cloud and Grid execution using Serpens, Nimrod/K and Globus actors. We also present real-life applications in computational chemistry, bioinformatics and computational physics to demonstrate the usage of different distributed computing capabilities of Kepler in executable workflows. We further analyze the differences of each approach and provide a guidance for their applications.",

keywords = "distributed execution, Kepler, scientific workflow",

author = "Marcin P{\l}{\'o}ciennik and Tomasz Zok and Ilkay Altintas and Jianwu Wang and Daniel Crawl and David Abramson and Frederic Imbeaux and Bernard Guillerminet and Marcos Lopez-Caniego and Plasencia, {Isabel Campos} and Wojciech Pych and Pawel Cieciel{\c a}g and Bartek Palak and Micha{\l} Owsiak and Yann Frauel",

year = "2013",

doi = "10.3233/FI-2013-947",

language = "English",

volume = "128",

pages = "281--302",

journal = "Fundamenta Informaticae",

issn = "0169-2968",

publisher = "Polskie Towarzystwo Matematyczne",

number = "3",

}

TY - JOUR

T1 - Approaches to distributed execution of scientific workflows in Kepler

AU - Płóciennik, Marcin

AU - Zok, Tomasz

AU - Altintas, Ilkay

AU - Wang, Jianwu

AU - Crawl, Daniel

AU - Abramson, David

AU - Imbeaux, Frederic

AU - Guillerminet, Bernard

AU - Lopez-Caniego, Marcos

AU - Plasencia, Isabel Campos

AU - Pych, Wojciech

AU - Cieciela̧g, Pawel

AU - Palak, Bartek

AU - Owsiak, Michał

AU - Frauel, Yann

PY - 2013

Y1 - 2013

N2 - The Kepler scientific workflow system enables creation, execution and sharing of workflows across a broad range of scientific and engineering disciplines while also facilitating remote and distributed execution of workflows. In this paper, we present and compare different approaches to distributed execution of workflows using the Kepler environment, including a distributed data-parallel framework using Hadoop and Stratosphere, and Cloud and Grid execution using Serpens, Nimrod/K and Globus actors. We also present real-life applications in computational chemistry, bioinformatics and computational physics to demonstrate the usage of different distributed computing capabilities of Kepler in executable workflows. We further analyze the differences of each approach and provide a guidance for their applications.

AB - The Kepler scientific workflow system enables creation, execution and sharing of workflows across a broad range of scientific and engineering disciplines while also facilitating remote and distributed execution of workflows. In this paper, we present and compare different approaches to distributed execution of workflows using the Kepler environment, including a distributed data-parallel framework using Hadoop and Stratosphere, and Cloud and Grid execution using Serpens, Nimrod/K and Globus actors. We also present real-life applications in computational chemistry, bioinformatics and computational physics to demonstrate the usage of different distributed computing capabilities of Kepler in executable workflows. We further analyze the differences of each approach and provide a guidance for their applications.

KW - distributed execution

KW - Kepler

KW - scientific workflow

UR - http://www.scopus.com/inward/record.url?scp=84887929995&partnerID=8YFLogxK

U2 - 10.3233/FI-2013-947

DO - 10.3233/FI-2013-947

M3 - Article

AN - SCOPUS:84887929995

VL - 128

SP - 281

EP - 302

JO - Fundamenta Informaticae

JF - Fundamenta Informaticae

SN - 0169-2968

IS - 3

ER -

Approaches to distributed execution of scientific workflows in Kepler

Abstract

Keywords

Access to Document

Other files and links

Cite this