Scalable Deployment of a LIGO Physics Application on Public Clouds: Workflow Engine and Resource Provisioning Techniques

Suraj Pandey, Letizia Sammut, Rodrigo N Calheiros, Andrew Melatos, Rajkumar Buyya

Research output: Chapter in Book/Report/Conference proceedingChapter (Book)Research

Abstract

Cloud computing has empowered users to provision virtually unlimited computational resources and are accessible over the Internet on demand. This makes Cloud computing a compelling technology that tackles the issues rising with the growing size and complexity of scientific applications, which are characterized by high variance in usage, large volume of data and high compute load, flash crowds, unpredictable load, and varying compute and storage requirements. In order to provide users an automated and scalable platform for hosting scientific workflow applications, while hiding the complexity of the underlying Cloud infrastructure, we present the design and implementation of a PaaS middleware solution along with resource provisioning techniques. We apply our PaaS solution to the data analysis pipeline of a physics application, a gravitational wave search, utilizing public Clouds. The system architecture, a load-balancing approach, and the system’s behavior over varying loads are detailed. The performance evaluation on scalability and load-balancing characteristics of the automated PaaS middleware demonstrates the feasibility and advantages of the approach over existing monolithic approaches.
Original languageEnglish
Title of host publicationCloud Computing for Data-Intensive Applications
EditorsXiaolin Li, Judy Qiu
Place of PublicationNew York
PublisherSpringer
Pages3-25
Number of pages22
ISBN (Electronic)978-1-4939-1905-5
ISBN (Print)978-1-4939-1904-8
DOIs
Publication statusPublished - 2014
Externally publishedYes

Cite this