AI for Energy Demand Forecasting

The Merantix Research Day is a yearly event where all of our Merantix ventures come together to share what they are working on and highlight technical achievements that have been made. This article series highlights a few of the talks given at the 2023 edition of our Research Day, and was written with the help of Open AI’s Whisper and ChatGPT.

A Journey into Energy Demand Forecasting

Konstantin Ditschuneit and Julien Siems, ML Engineers at Merantix Momentum, shared their experience in winning a Kaggle challenge on energy demand forecasting. With limited experience in time series and energy data, they relied on experimentation and iterative development. In this blog post we want to give some more insights into their approach.

The Kaggle Challenge

The Kaggle challenge, organised by Hessian AI, aimed to develop an algorithm that could forecast the residual load of a factory. Konstantin and Julien began their journey in December 2022, and despite having little prior experience in the field, they were determined to learn and adapt as they progressed. The data set they worked with was collected by Hessian AI from a factory in Hessen, Germany. It included various features on sun, wind, and temperature for each time step.

Model Selection and Development

Initially, the duo experimented with models such as XGBoost, LGBoost, and Facebook's Prophet library. However, they eventually settled on NeuralProphet, an extension of Prophet with an autoregressive component and a PyTorch-based implementation. NeuralProphet is an additive model that combines several components such as trend, events, seasonality, and additional regressors.

Technical Background

NeuralProphet is based on the additive time series decomposition, where the output is computed by summing the individual components for each time step. The trend component accounts for the overall direction of the data, while the seasonality component captures periodic fluctuations. The event component represents irregular occurrences such as holidays or worker strikes. Additional regressors include external factors like weather data, which can provide valuable context for the model.

Data Improvement and Model Refinement

To improve their model's performance, Konstantin and Julien incorporated additional weather data from all over Hessen. They scraped this data from the Deutsche Wetterdienst, which openly provided weather data such as wind, solar temperature, and extreme wind. By including weather data from multiple stations, they significantly reduced the error and created a more robust model.

Next, they replaced NeuralProphet's linear regression component with a deep network, allowing for a more complex integration of the weather data. They discovered that the placement of normalization layers within this multi-layer perceptron played a crucial role in their model's performance.

Hyperparameter Optimization and Visualization

Utilizing Optuna for hyperparameter optimization and Weights & Biases for visualization, the duo further improved their model's performance. They trained numerous models and analyzed their results to identify which hyperparameters had the most significant impact on their model's success.

Ensemble Techniques and Weighted Submissions

To further enhance their model's performance, Julien and Konstantin employed ensemble techniques, combining the predictions of multiple models. They then utilized a weighted submissions approach, which involved assigning different weights to individual submissions to optimize the final error score

Outlook

The success in the Kaggle challenge demonstrates the power of iterative development and experimentation in the machine learning process. By incorporating various models, data sources, and optimization techniques, they managed to create a powerful forecasting algorithm called MLP Prophet. This winning model will now be used in their longer-term research project, FlexGUIde, showcasing the real-world impact of their efforts.