Insights on our models and their performances during H1 2021

20 July 2021

In the past few months, markets have been all over the place. Let’s take France for example: We have had a decreasing demand since January, as we should have. However, accompanying this decrease in demand, was a steady rise in prices. This inverse correlation, which is rather uncommon, could be partially explained by the sharp rise in CO2 and gas prices. Faced with uncertainties, the markets reacted with a stronger than usual volatility.

Power prices in France over H1 2021, source ENTSO-e

Power demand in France over H1 2021, source ENTSO-e

In our last newsletter, we went over our performance in the past two years, which was overall very satisfying. Indeed, while our AI models do not pretend to outperform all traditional models (yet), what we lose in performance compared to the best Stack / Delta models, we gain 10x in flexibility, with the ability to omit or add data sets into the models quite easily. This flexibility allowed us to quickly adapt to extraordinary circumstances, from the first COVID lockdown restrictions back in March of last year to today.

While we did fairly well for the past two years, how did we perform during the first half of this tumultuous year?

Let’s cut to the chase: in regard to prices and in terms of absolute numbers, we did slightly worse. However, given the current highly volatile environment, the results were better than it seemed, as our forecasts still remained competitive and offered some good insights on market movements.  

We started the year with a daily MAE (Mean Absolute Error) around 3€, slowly decreasing in the following months, sharply rising in April and especially in May. This was mainly due to the meteoric rise in carbon and gas prices, as mentioned above. Our model, which learns from the past, had difficulties capturing the situation. However, as the model is constantly learning and improving, it quickly adapted and overcame the issue, with its performance going back to more satisfactory levels.

If we look at our Week-Ahead forecasts, we have maintained a competitive MAE, while providing a strong edge in terms of directional forecasts.

Our short-term forecasts are based on daily comprehensive weather forecasts (among other inputs). We get the data from two weather centers both providing operational and ensemble forecast, with slight difference in resolution:

  • ECMWF (European Centre for Medium-Range Weather Forecasts) proving a high resolution OP forecast at 0.1° and ENS at 0.2° granularity
  • NOAA (US National Operational Model Archive) providing GFS forecast, with OP at 0.25° and ENS at 0.5°

We compute all 30 to 50 different weather scenarios of ENS forecast, allowing us to display the mean, but also various quantiles of the results on our platform (Q5/ Q25 / Q50 / Q75 / Q95).

When it comes to prices, EC OP usually performs the best, which is why we base our backtests on EC OP. In fact, we suggest looking at EC OP from Day-Ahead to D+3, and at EC ENS from D+4 onwards.

The reason is quite simple. Our ENS models are showing the mean of all the probabilistic forecasts we generated. As weather forecasts are less accurate the further the horizon, it is only logical that the forecast generated by the mean of all the different possibilities would be, on average, the most accurate when you look at a few days out.  Our GFS forecasts are still relevant as they significantly outperform our EC forecasts on some occasions.

Our forecasts are not limited to price only. We also generate fundamental forecasts, which are just as fast. If we look at the past 30 days, our FR demand forecast averages a 1.57% hourly MAPE (Mean Absolute Percentage Error). The daily MAPE is almost twice as small.

Power demand forecasts in France from the 21st of June until the 20th of July 2021, by COR-e models

HRES – Hourly MAE : 685 MW, Daily MAE 385MW, Hourly MAPE 1.57%, Daily MAPE 0.87%

As for our wind & solar production forecasts, we are usually off by a few hundreds of MW only. The combination of all these forecasts makes for a great decision assisting tool. The key resides in scoring each weather model, to see which one give the most insight during specific situations and conditions.

At COR-e, we give a lot of value to transparency, as it is the most important step in earning trust. Whatever our performances are, we will always be proud to share them with our current and future clients. We will always be open to discussion and willing to learn from your feedback.

Kenan Saray
Sales Executive, COR-e

COR-e Logo

393 rue Nicéphore Niepce

83400 Hyères, France

client@cor-e.tech

© COR-e 2021