Towards rigorous validation of energy optimisation experiments

Mahmoud A. Bokhari, Brad Alexander, Markus Wagner

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

6 Citations (Scopus)


The optimisation of software energy consumption is of growing importance across all scales of modern computing, i.e., from embedded systems to data-centres. Practitioners in the field of Search-Based Software Engineering and Genetic Improvement of Software acknowledge that optimising software energy consumption is difficult due to noisy and expensive fitness evaluations. However, it is apparent from results to date that more progress needs to be made in rigorously validating optimisation results. This problem is pressing because modern computing platforms have highly complex and variable behaviour with respect to energy consumption. To compare solutions fairly we propose in this paper a new validation approach called R3-validation which exercises software variants in a rotated-round-robin order. Using a case study, we present an in-depth analysis of the impacts of changing system states on software energy usage, and we show how R3-validation mitigates these. We compare it with current validation approaches across multiple devices and operating systems, and we show that it aligns best with actual platform behaviour.

Original languageEnglish
Title of host publicationProceedings of the 2020 Genetic and Evolutionary Computation Conference
EditorsCarlos Coello Coello
Place of PublicationNew York NY USA
PublisherAssociation for Computing Machinery (ACM)
Number of pages9
ISBN (Electronic)9781450371285
Publication statusPublished - 2020
Externally publishedYes
EventThe Genetic and Evolutionary Computation Conference 2020 - Cancun, Mexico
Duration: 8 Jul 202012 Jul 2020
Conference number: 22nd (Proceedings)


ConferenceThe Genetic and Evolutionary Computation Conference 2020
Abbreviated titleGECCO 2020
Internet address


  • Android
  • Energy consumption
  • Mobile applications
  • Non-functional properties

Cite this