Regularization in hierarchical time series forecasting with application to electricity smart meter data

Souhaib Ben Taieb, Jiafan Yu, Mateus Neves Barreto, Ram Rajagopal

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

Abstract

Accurate electricity demand forecast plays a key role in sustainable power systems. It enables better decision making in the planning of electricity generation and distribution for many use cases. The electricity demand data can often be represented in a hierarchical structure. For example, the electricity consumption of a whole country could be disaggregated by states, cities, and households. Hierarchical forecasts require not only good prediction accuracy at each level of the hierarchy, but also the consistency between different levels. State-of-the-art hierarchical forecasting methods usually apply adjustments on the individual level forecasts to satisfy the aggregation constraints. However, the high-dimensionality of the unpenalized regression problem and the estimation errors in the high-dimensional error covariance matrix can lead to increased variability in the revised forecasts with poor prediction performance. In order to provide more robustness to estimation errors in the adjustments, we present a new hierarchical forecasting algorithm that computes sparse adjustments while still preserving the aggregation constraints. We formulate the problem as a high-dimensional penalized regression, which can be efficiently solved using cyclical coordinate descent methods. We also conduct experiments using a large-scale hierarchical electricity demand data. The results confirm the effectiveness of our approach compared to state-of-the-art hierarchical forecasting methods, in both the sparsity of the adjustments and the prediction accuracy. The proposed approach to hierarchical forecasting could be useful for energy generation including solar and wind energy, as well as numerous other applications.
Original languageEnglish
Title of host publicationProceedings of the Thirty-First AAAI Conference on Artificial Intelligence (AAAI-17)
EditorsSatinder Singh, Shaul Markovitch
Place of PublicationPalto Alto CA USA
PublisherAssociation for the Advancement of Artificial Intelligence (AAAI)
Pages4474-4480
Number of pages7
Publication statusPublished - 2017
EventAAAI Conference on Artificial Intelligence 2017 - Hilton San Francisco Union Square, San Francisco, United States of America
Duration: 4 Feb 201710 Feb 2017
Conference number: 31st
http://www.aaai.org/Conferences/AAAI/aaai17.php

Conference

ConferenceAAAI Conference on Artificial Intelligence 2017
Abbreviated titleAAAI 2017
CountryUnited States of America
CitySan Francisco
Period4/02/1710/02/17
Internet address

Cite this

Ben Taieb, S., Yu, J., Barreto, M. N., & Rajagopal, R. (2017). Regularization in hierarchical time series forecasting with application to electricity smart meter data. In S. Singh, & S. Markovitch (Eds.), Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence (AAAI-17) (pp. 4474-4480). Palto Alto CA USA: Association for the Advancement of Artificial Intelligence (AAAI).
Ben Taieb, Souhaib ; Yu, Jiafan ; Barreto, Mateus Neves ; Rajagopal, Ram. / Regularization in hierarchical time series forecasting with application to electricity smart meter data. Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence (AAAI-17). editor / Satinder Singh ; Shaul Markovitch. Palto Alto CA USA : Association for the Advancement of Artificial Intelligence (AAAI), 2017. pp. 4474-4480
@inproceedings{ed7cca056c594dbb8e15eadb1a8397d3,
title = "Regularization in hierarchical time series forecasting with application to electricity smart meter data",
abstract = "Accurate electricity demand forecast plays a key role in sustainable power systems. It enables better decision making in the planning of electricity generation and distribution for many use cases. The electricity demand data can often be represented in a hierarchical structure. For example, the electricity consumption of a whole country could be disaggregated by states, cities, and households. Hierarchical forecasts require not only good prediction accuracy at each level of the hierarchy, but also the consistency between different levels. State-of-the-art hierarchical forecasting methods usually apply adjustments on the individual level forecasts to satisfy the aggregation constraints. However, the high-dimensionality of the unpenalized regression problem and the estimation errors in the high-dimensional error covariance matrix can lead to increased variability in the revised forecasts with poor prediction performance. In order to provide more robustness to estimation errors in the adjustments, we present a new hierarchical forecasting algorithm that computes sparse adjustments while still preserving the aggregation constraints. We formulate the problem as a high-dimensional penalized regression, which can be efficiently solved using cyclical coordinate descent methods. We also conduct experiments using a large-scale hierarchical electricity demand data. The results confirm the effectiveness of our approach compared to state-of-the-art hierarchical forecasting methods, in both the sparsity of the adjustments and the prediction accuracy. The proposed approach to hierarchical forecasting could be useful for energy generation including solar and wind energy, as well as numerous other applications.",
author = "{Ben Taieb}, Souhaib and Jiafan Yu and Barreto, {Mateus Neves} and Ram Rajagopal",
year = "2017",
language = "English",
pages = "4474--4480",
editor = "Satinder Singh and Shaul Markovitch",
booktitle = "Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence (AAAI-17)",
publisher = "Association for the Advancement of Artificial Intelligence (AAAI)",
address = "United States of America",

}

Ben Taieb, S, Yu, J, Barreto, MN & Rajagopal, R 2017, Regularization in hierarchical time series forecasting with application to electricity smart meter data. in S Singh & S Markovitch (eds), Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence (AAAI-17). Association for the Advancement of Artificial Intelligence (AAAI), Palto Alto CA USA, pp. 4474-4480, AAAI Conference on Artificial Intelligence 2017, San Francisco, United States of America, 4/02/17.

Regularization in hierarchical time series forecasting with application to electricity smart meter data. / Ben Taieb, Souhaib; Yu, Jiafan; Barreto, Mateus Neves; Rajagopal, Ram.

Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence (AAAI-17). ed. / Satinder Singh; Shaul Markovitch. Palto Alto CA USA : Association for the Advancement of Artificial Intelligence (AAAI), 2017. p. 4474-4480.

Research output: Chapter in Book/Report/Conference proceedingConference PaperResearchpeer-review

TY - GEN

T1 - Regularization in hierarchical time series forecasting with application to electricity smart meter data

AU - Ben Taieb, Souhaib

AU - Yu, Jiafan

AU - Barreto, Mateus Neves

AU - Rajagopal, Ram

PY - 2017

Y1 - 2017

N2 - Accurate electricity demand forecast plays a key role in sustainable power systems. It enables better decision making in the planning of electricity generation and distribution for many use cases. The electricity demand data can often be represented in a hierarchical structure. For example, the electricity consumption of a whole country could be disaggregated by states, cities, and households. Hierarchical forecasts require not only good prediction accuracy at each level of the hierarchy, but also the consistency between different levels. State-of-the-art hierarchical forecasting methods usually apply adjustments on the individual level forecasts to satisfy the aggregation constraints. However, the high-dimensionality of the unpenalized regression problem and the estimation errors in the high-dimensional error covariance matrix can lead to increased variability in the revised forecasts with poor prediction performance. In order to provide more robustness to estimation errors in the adjustments, we present a new hierarchical forecasting algorithm that computes sparse adjustments while still preserving the aggregation constraints. We formulate the problem as a high-dimensional penalized regression, which can be efficiently solved using cyclical coordinate descent methods. We also conduct experiments using a large-scale hierarchical electricity demand data. The results confirm the effectiveness of our approach compared to state-of-the-art hierarchical forecasting methods, in both the sparsity of the adjustments and the prediction accuracy. The proposed approach to hierarchical forecasting could be useful for energy generation including solar and wind energy, as well as numerous other applications.

AB - Accurate electricity demand forecast plays a key role in sustainable power systems. It enables better decision making in the planning of electricity generation and distribution for many use cases. The electricity demand data can often be represented in a hierarchical structure. For example, the electricity consumption of a whole country could be disaggregated by states, cities, and households. Hierarchical forecasts require not only good prediction accuracy at each level of the hierarchy, but also the consistency between different levels. State-of-the-art hierarchical forecasting methods usually apply adjustments on the individual level forecasts to satisfy the aggregation constraints. However, the high-dimensionality of the unpenalized regression problem and the estimation errors in the high-dimensional error covariance matrix can lead to increased variability in the revised forecasts with poor prediction performance. In order to provide more robustness to estimation errors in the adjustments, we present a new hierarchical forecasting algorithm that computes sparse adjustments while still preserving the aggregation constraints. We formulate the problem as a high-dimensional penalized regression, which can be efficiently solved using cyclical coordinate descent methods. We also conduct experiments using a large-scale hierarchical electricity demand data. The results confirm the effectiveness of our approach compared to state-of-the-art hierarchical forecasting methods, in both the sparsity of the adjustments and the prediction accuracy. The proposed approach to hierarchical forecasting could be useful for energy generation including solar and wind energy, as well as numerous other applications.

M3 - Conference Paper

SP - 4474

EP - 4480

BT - Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence (AAAI-17)

A2 - Singh, Satinder

A2 - Markovitch, Shaul

PB - Association for the Advancement of Artificial Intelligence (AAAI)

CY - Palto Alto CA USA

ER -

Ben Taieb S, Yu J, Barreto MN, Rajagopal R. Regularization in hierarchical time series forecasting with application to electricity smart meter data. In Singh S, Markovitch S, editors, Proceedings of the Thirty-First AAAI Conference on Artificial Intelligence (AAAI-17). Palto Alto CA USA: Association for the Advancement of Artificial Intelligence (AAAI). 2017. p. 4474-4480