CPU frequency tuning to improve energy efficiency of mapreduce systems

Nidhi Tiwari; Umesh Bellur; Santonu Sarkar; Maria Indrawan

doi:10.1109/ICPADS.2016.0135

CPU frequency tuning to improve energy efficiency of mapreduce systems

Nidhi Tiwari
, Umesh Bellur
, Santonu Sarkar
, Maria Indrawan

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › Research

5 Citations (Scopus)

Abstract

Energy efficiency is a major concern in today's data centers that house large scale distributed processing systems such as data parallel MapReduce clusters. Modern power aware systems utilize the dynamic voltage and frequency scaling mechanism available in processors to manage the energy consumption. In this paper, we initially characterize the energy efficiency of MapReduce jobs with respect to built-in power governors. Our analysis indicates that while a built-in power governor provides the best energy efficiency for a job that is CPU as well as IO intensive, a common CPU-frequency across the cluster provides best the energy efficiency for other types of jobs. In order to identify this optimal frequency setting, we derive energy and performance models for MapReduce jobs on a HPC cluster and validate these models experimentally on different platforms. We demonstrate how these models can be used to improve energy efficiency of the machine learning MapReduce applications running on the Yarn platform. The execution of jobs at their optimal frequencies improves the energy efficiency by average 25% over the default governor setting. In case of mixed workloads, the energy efficiency improves by up to 10% when we use an optimal CPU-frequency across the cluster.

Original language	English
Title of host publication	Proceedings of 2016 IEEE 22nd International Conference on Parallel and Distributed Systems (ICPADS 2016)
Subtitle of host publication	13-16 December 2016, Wuhan, China
Editors	Xiaofei Liao, Robert Lovas, Xipeng Shen, Ran Zheng
Place of Publication	Piscataway, NJ
Publisher	IEEE, Institute of Electrical and Electronics Engineers
Pages	1015-1022
Number of pages	8
ISBN (Electronic)	9781509044573
ISBN (Print)	9781509053827
DOIs	https://doi.org/10.1109/ICPADS.2016.0135
Publication status	Published - Dec 2016
Event	International Conference on Parallel and Distributed Systems 2016 - Wuhan Hubei, China Duration: 13 Dec 2016 → 16 Dec 2016 Conference number: 22nd http://ieeexplore.ieee.org/xpl/mostRecentIssue.jsp?punumber=7822825 (Conference Proceedings)

Conference

Conference	International Conference on Parallel and Distributed Systems 2016
Abbreviated title	ICPADS 2016
Country/Territory	China
City	Wuhan Hubei
Period	13/12/16 → 16/12/16
Internet address	http://ieeexplore.ieee.org/xpl/mostRecentIssue.jsp?punumber=7822825 (Conference Proceedings)

UN SDGs

This output contributes to the following UN Sustainable Development Goals (SDGs)

SDG 7 Affordable and Clean Energy

Keywords

CPU-Frequency Tuning
Distributed Computing
Energy Efficiency
MapReduce
Predictive Models

Access to Document

10.1109/ICPADS.2016.0135

Cite this

Tiwari, N., Bellur, U., Sarkar, S., & Indrawan, M. (2016). CPU frequency tuning to improve energy efficiency of mapreduce systems. In X. Liao, R. Lovas, X. Shen, & R. Zheng (Eds.), Proceedings of 2016 IEEE 22nd International Conference on Parallel and Distributed Systems (ICPADS 2016): 13-16 December 2016, Wuhan, China (pp. 1015-1022). Article 7823850 IEEE, Institute of Electrical and Electronics Engineers. https://doi.org/10.1109/ICPADS.2016.0135

Tiwari, Nidhi ; Bellur, Umesh ; Sarkar, Santonu et al. / CPU frequency tuning to improve energy efficiency of mapreduce systems. Proceedings of 2016 IEEE 22nd International Conference on Parallel and Distributed Systems (ICPADS 2016): 13-16 December 2016, Wuhan, China. editor / Xiaofei Liao ; Robert Lovas ; Xipeng Shen ; Ran Zheng. Piscataway, NJ : IEEE, Institute of Electrical and Electronics Engineers, 2016. pp. 1015-1022

@inproceedings{9e5bd0c47adf4a548b0f7e08b55444c9,

title = "CPU frequency tuning to improve energy efficiency of mapreduce systems",

abstract = "Energy efficiency is a major concern in today's data centers that house large scale distributed processing systems such as data parallel MapReduce clusters. Modern power aware systems utilize the dynamic voltage and frequency scaling mechanism available in processors to manage the energy consumption. In this paper, we initially characterize the energy efficiency of MapReduce jobs with respect to built-in power governors. Our analysis indicates that while a built-in power governor provides the best energy efficiency for a job that is CPU as well as IO intensive, a common CPU-frequency across the cluster provides best the energy efficiency for other types of jobs. In order to identify this optimal frequency setting, we derive energy and performance models for MapReduce jobs on a HPC cluster and validate these models experimentally on different platforms. We demonstrate how these models can be used to improve energy efficiency of the machine learning MapReduce applications running on the Yarn platform. The execution of jobs at their optimal frequencies improves the energy efficiency by average 25\% over the default governor setting. In case of mixed workloads, the energy efficiency improves by up to 10\% when we use an optimal CPU-frequency across the cluster.",

keywords = "CPU-Frequency Tuning, Distributed Computing, Energy Efficiency, MapReduce, Predictive Models",

author = "Nidhi Tiwari and Umesh Bellur and Santonu Sarkar and Maria Indrawan",

year = "2016",

month = dec,

doi = "10.1109/ICPADS.2016.0135",

language = "English",

isbn = "9781509053827",

pages = "1015--1022",

editor = "Liao, \{Xiaofei \} and Lovas, \{Robert \} and Shen, \{Xipeng \} and Zheng, \{Ran \}",

booktitle = "Proceedings of 2016 IEEE 22nd International Conference on Parallel and Distributed Systems (ICPADS 2016)",

publisher = "IEEE, Institute of Electrical and Electronics Engineers",

address = "United States of America",

note = "International Conference on Parallel and Distributed Systems 2016, ICPADS 2016 ; Conference date: 13-12-2016 Through 16-12-2016",

url = "http://ieeexplore.ieee.org/xpl/mostRecentIssue.jsp?punumber=7822825",

}

Tiwari, N, Bellur, U, Sarkar, S & Indrawan, M 2016, CPU frequency tuning to improve energy efficiency of mapreduce systems. in X Liao, R Lovas, X Shen & R Zheng (eds), Proceedings of 2016 IEEE 22nd International Conference on Parallel and Distributed Systems (ICPADS 2016): 13-16 December 2016, Wuhan, China., 7823850, IEEE, Institute of Electrical and Electronics Engineers, Piscataway, NJ, pp. 1015-1022, International Conference on Parallel and Distributed Systems 2016, Wuhan Hubei, China, 13/12/16. https://doi.org/10.1109/ICPADS.2016.0135

CPU frequency tuning to improve energy efficiency of mapreduce systems. / Tiwari, Nidhi; Bellur, Umesh; Sarkar, Santonu et al.
Proceedings of 2016 IEEE 22nd International Conference on Parallel and Distributed Systems (ICPADS 2016): 13-16 December 2016, Wuhan, China. ed. / Xiaofei Liao; Robert Lovas; Xipeng Shen; Ran Zheng. Piscataway, NJ: IEEE, Institute of Electrical and Electronics Engineers, 2016. p. 1015-1022 7823850.

Research output: Chapter in Book/Report/Conference proceeding › Conference Paper › Research

TY - GEN

T1 - CPU frequency tuning to improve energy efficiency of mapreduce systems

AU - Tiwari, Nidhi

AU - Bellur, Umesh

AU - Sarkar, Santonu

AU - Indrawan, Maria

N1 - Conference code: 22nd

PY - 2016/12

Y1 - 2016/12

N2 - Energy efficiency is a major concern in today's data centers that house large scale distributed processing systems such as data parallel MapReduce clusters. Modern power aware systems utilize the dynamic voltage and frequency scaling mechanism available in processors to manage the energy consumption. In this paper, we initially characterize the energy efficiency of MapReduce jobs with respect to built-in power governors. Our analysis indicates that while a built-in power governor provides the best energy efficiency for a job that is CPU as well as IO intensive, a common CPU-frequency across the cluster provides best the energy efficiency for other types of jobs. In order to identify this optimal frequency setting, we derive energy and performance models for MapReduce jobs on a HPC cluster and validate these models experimentally on different platforms. We demonstrate how these models can be used to improve energy efficiency of the machine learning MapReduce applications running on the Yarn platform. The execution of jobs at their optimal frequencies improves the energy efficiency by average 25% over the default governor setting. In case of mixed workloads, the energy efficiency improves by up to 10% when we use an optimal CPU-frequency across the cluster.

AB - Energy efficiency is a major concern in today's data centers that house large scale distributed processing systems such as data parallel MapReduce clusters. Modern power aware systems utilize the dynamic voltage and frequency scaling mechanism available in processors to manage the energy consumption. In this paper, we initially characterize the energy efficiency of MapReduce jobs with respect to built-in power governors. Our analysis indicates that while a built-in power governor provides the best energy efficiency for a job that is CPU as well as IO intensive, a common CPU-frequency across the cluster provides best the energy efficiency for other types of jobs. In order to identify this optimal frequency setting, we derive energy and performance models for MapReduce jobs on a HPC cluster and validate these models experimentally on different platforms. We demonstrate how these models can be used to improve energy efficiency of the machine learning MapReduce applications running on the Yarn platform. The execution of jobs at their optimal frequencies improves the energy efficiency by average 25% over the default governor setting. In case of mixed workloads, the energy efficiency improves by up to 10% when we use an optimal CPU-frequency across the cluster.

KW - CPU-Frequency Tuning

KW - Distributed Computing

KW - Energy Efficiency

KW - MapReduce

KW - Predictive Models

UR - https://www.scopus.com/pages/publications/85018480554

U2 - 10.1109/ICPADS.2016.0135

DO - 10.1109/ICPADS.2016.0135

M3 - Conference Paper

SN - 9781509053827

SP - 1015

EP - 1022

BT - Proceedings of 2016 IEEE 22nd International Conference on Parallel and Distributed Systems (ICPADS 2016)

A2 - Liao, Xiaofei

A2 - Lovas, Robert

A2 - Shen, Xipeng

A2 - Zheng, Ran

PB - IEEE, Institute of Electrical and Electronics Engineers

CY - Piscataway, NJ

T2 - International Conference on Parallel and Distributed Systems 2016

Y2 - 13 December 2016 through 16 December 2016

ER -

Tiwari N, Bellur U, Sarkar S, Indrawan M. CPU frequency tuning to improve energy efficiency of mapreduce systems. In Liao X, Lovas R, Shen X, Zheng R, editors, Proceedings of 2016 IEEE 22nd International Conference on Parallel and Distributed Systems (ICPADS 2016): 13-16 December 2016, Wuhan, China. Piscataway, NJ: IEEE, Institute of Electrical and Electronics Engineers. 2016. p. 1015-1022. 7823850 doi: 10.1109/ICPADS.2016.0135

CPU frequency tuning to improve energy efficiency of mapreduce systems

Abstract

Conference

UN SDGs

Keywords

Access to Document

Other files and links

Cite this