CPU frequency tuning to improve energy efficiency of mapreduce systems

Nidhi Tiwari, Umesh Bellur, Santonu Sarkar, Maria Indrawan

    Research output: Chapter in Book/Report/Conference proceedingConference PaperResearch

    5 Citations (Scopus)


    Energy efficiency is a major concern in today's data centers that house large scale distributed processing systems such as data parallel MapReduce clusters. Modern power aware systems utilize the dynamic voltage and frequency scaling mechanism available in processors to manage the energy consumption. In this paper, we initially characterize the energy efficiency of MapReduce jobs with respect to built-in power governors. Our analysis indicates that while a built-in power governor provides the best energy efficiency for a job that is CPU as well as IO intensive, a common CPU-frequency across the cluster provides best the energy efficiency for other types of jobs. In order to identify this optimal frequency setting, we derive energy and performance models for MapReduce jobs on a HPC cluster and validate these models experimentally on different platforms. We demonstrate how these models can be used to improve energy efficiency of the machine learning MapReduce applications running on the Yarn platform. The execution of jobs at their optimal frequencies improves the energy efficiency by average 25% over the default governor setting. In case of mixed workloads, the energy efficiency improves by up to 10% when we use an optimal CPU-frequency across the cluster.

    Original languageEnglish
    Title of host publicationProceedings of 2016 IEEE 22nd International Conference on Parallel and Distributed Systems (ICPADS 2016)
    Subtitle of host publication13-16 December 2016, Wuhan, China
    EditorsXiaofei Liao, Robert Lovas, Xipeng Shen, Ran Zheng
    Place of PublicationPiscataway, NJ
    PublisherIEEE, Institute of Electrical and Electronics Engineers
    Number of pages8
    ISBN (Electronic)9781509044573
    ISBN (Print)9781509053827
    Publication statusPublished - Dec 2016
    EventInternational Conference on Parallel and Distributed Systems 2016 - Wuhan Hubei, China
    Duration: 13 Dec 201616 Dec 2016
    Conference number: 22nd
    http://ieeexplore.ieee.org/xpl/mostRecentIssue.jsp?punumber=7822825 (Conference Proceedings)


    ConferenceInternational Conference on Parallel and Distributed Systems 2016
    Abbreviated titleICPADS 2016
    CityWuhan Hubei
    Internet address


    • CPU-Frequency Tuning
    • Distributed Computing
    • Energy Efficiency
    • MapReduce
    • Predictive Models

    Cite this