High-Resolution Peak Demand Estimation Using Generalized Additive Models and Deep Neural Networks

.title[
# High-Resolution Peak Demand Estimation Using Generalized Additive Models and Deep Neural Networks
]
.subtitle[
## Jonathan Berrisch, Michał Narajewski, Florian Ziel
]
.author[
### University of Duisburg-Essen
]
.date[
### 2022-07-12
]

---

# Motivation

- Organized by Western Power Distribution and Catapult Energy Systems
- Does limited high-resolution monitoring help estimate future high-resolution peak loads?

The Objective:
- Estimate minimum and maximum electricity load values (one-minute resolution)
- Given data with only a 30-minute resolution
- One single substation, every half-hour of September 2021

Data:
- From Nov. 2019 to Sept. 2021 (30-minute resolution)
- MERRA-2 weather reanalysis data from five locations close to the substation

---

# Motivation

### Location Overview

---
  
# Motivation

### Data Overview

---

# Data

Targets: `$\Delta^{\min}_t$` and `$\Delta^{\max}_t$`

Possible explanatory variables:

The *half-hourly* load: `$L_t$`

Discrete second order central difference (DSOCD):

`$L_t'' = L_{t-1}-2L_t+L_{t+1}$`

Deterministic components (to capture potential seasonal characteristics)
- Daily `$D_t$` number of hours in a day
- Weekly `$W_t$` number of hours in a week
- Annual  `$A_t$` number of hours in a meteorological year with `$365.24$` days
]

Weather Inputs: Temperature, Windspeed (North / East), Solar, Humidity

The Figure (next slide) shows how these variables correlate.
- Lower triangle:
  - Pearson's correlation
- Upper triangle:
  - Distance correlation

Distance correlation: non-linear dependency measure that takes values in `$[0, 1]$` and characterizes stochastic independence <a id='cite-szekely2007measuring'></a><a href='#bib-szekely2007measuring'>Székely, Rizzo, and Bakirov (2007)</a>.

]

---

# Correlation

---

# Data

---

# Feature Set and Models

Selected features based on preliminary analysis:

| Variable type  | Included feature  | Number  |
|---|---|---|
| Lagged load  | `$L_{t-1}$`, `$L_t$`, `$L_{t+1}$`  |  3 |
| Lagged DSOCD load  |  `$\widetilde{L}_{t-4}'',\ldots , \widetilde{L}_{t+4}''$` | 9  |
| Weather inputs  | Temp, Solar, WindN, WindE, Press, Humid  | 6  |
| Seasonal inputs  |  `$D_t$`, `$W_t$`, `$A_t$` | 3  |

]

Considered Models:

- Generalized Additive Model (GAM)
- Multilayer Perceptron Network (MLP)
- Combinations of the above

Competition Benchmark:

`$$L_t^{\max} = L_t$$`
`$$L_t^{\min} = L_t$$`

]

---

# Modelling Approach: GAM

### Generalized Additive Model (GAM)

`\begin{equation}
	\Delta^{m}_t = \sum_{i=1}^L f_i( X_{t,1}, \ldots, X_{t,N}) + \epsilon_t
	\label{eq_GAM_gen}
\end{equation}`
where `$m\in\{\min , \max\}$`

Traditional framework using cubic B-splines

*GAM.full* Specification (with all 2-way interactions):

`\begin{align}
	\Delta^{m}_t =& \sum_{i=1}^N b_{k_0}( X_{t,i} ) + \nonumber \\
  &\sum_{i=1}^N \sum_{j=1,j>i}^N b_{k_1,k_2}( X_{t,i}, X_{t,j} ) +\epsilon_t
	\label{eq_big_GAM}
\end{align}`

]

`$b_{k_0}$` and `$b_{k_1,k_2}$` denote univariate and bivariate splines with `$k_0$`, and `$(k_1,k_2)$` knots.
Tensor interaction splines -> only capture the joint effects.

We set `$k_0=27$` and `$k_1=k_2=9$`. Thus, linear terms are specified by `$27$` parameters and bivariate terms by `$81$` parameters.

*GAM.red*:

Interactions with `$L_t$`, `$\widetilde{L}_t''$` and `$\text{Solar}_t$`.

*GAM.simple*:
`\begin{align}
	\Delta^{m}_t =  b_{k_0}( L_t ) +  b_{k_0}( \widetilde{L}_t'') + b_{k_0}( \text{Solar}_t )
	+\epsilon
	\label{eq_simple_GAM}
\end{align}`

]

---

# Modelling Approach: MLP

Multilayer Perceptron Network:
<img src="data:image/png;base64,#fig/nn_fig.svg" width="500px" style="display: block; margin: auto;" />

Hyperparameters are tuned using OPTUNA Python package <a id='cite-akiba2019optuna'></a><a href='#bib-akiba2019optuna'>Akiba, Sano, Yanase, Ohta, and Koyama (2019)</a>

]

.pull-right[
- Input feature selection
- Number of hidden layers -- either 2 or 3
- Dropout layer -- whether to use it after the input layer and, if yes, at what rate.
- Activation functions in the hidden layers: elu, relu, sigmoid, softmax, softplus, and tanh
- Number of neurons in the hidden layer drawn on an exp-scale from `$[4,128]$`
- `$L_1$` regularization -- whether to use it on the hidden layers and, if yes, at what rate.
- Learning rate for the Adam optimization algorithm drawn on an exp-scale from `$\left(10^{-5}, 10^{-1}\right)$` interval
]

---

# GAM Parameter Significance

|		Min                             | EDF                         | F    | Max                             | EDF  | F    |
| --- |
|		`$L_t$`                                    | 9.5                         | 8.9  | `$L_t$`                                    | 10.8 | 17.6 |
|		`$\widetilde{L}_{t-1}''$`                     | 8.8                         | 4.4  | `$\widetilde{L}_{t-2}''$`                     | 6.7  | 4.9  |
|		`$\widetilde{L}_{t+1}''$`                     | 8.4                         | 4.8  | `$L_{t-1}$`                                | 6.3  | 6.9  |
|		`$\widetilde{L}_{t}''$`                       | 8.2                         | 4.6  | `$L_{t+1}$`                                | 5.4  | 5.0  |
|		Humid                                    | 6.1                         | 3.7  | `$D_t$`                                    | 4.8  | 3.1  |
|		WindE                                    | 5.1                         | 8.9  | Temp                                     | 4.1  | 11.5 |
|		`$A_t$`                                    | 4.7                         | 4.9  | Solar                                    | 4.0  | 5.5  |
|		WindN                                    | 4.2                         | 8.5  | WindE                                    | 3.8  | 5.8  |
|		Temp                                     | 3.5                         | 3.6  | `$\widetilde{L}_{t+4}''$`                     | 3.2  | 2.1  |
|		`$L_{t-1}$`                                | 3.3                         | 1.3  | `$\widetilde{L}_{t+1}''$`                     | 3.1  | 2.2  |

]

|		Min                             | EDF                         | F    | Max                             | EDF  | F    |
| --- |
|		`$L_t$`, `$A_t$`                             | 26.7                        | 7.8  | `$L_t$`, `$A_t$`                             | 37.8 | 28.1 |
|		`$L_t$`, Solar                             | 20.7                        | 11.2 | `$L_t$`, `$D_t$`                             | 25.1 | 20.6 |
|		`$L_t$`, `$D_t$`                             | 18.3                        | 8.2  | Solar, `$A_t$`                             | 15.4 | 6.4  |
|		`$L_t$`, `$L_{t+1}$`                         | 17.1                        | 4.0  | `$\widetilde{L}_{t}''$`, `$\widetilde{L}_{t-1}''$` | 14.9 | 6.5  |
|		`$\widetilde{L}_{t}''$`, `$\widetilde{L}_{t-1}''$` | 13.5                        | 2.5  | Solar, `$D_t$`                             | 11.0 | 8.5  |
|		`$L_t$`, Temp                              | 13.3                        | 3.1  | `$\widetilde{L}_{t}''$`, `$\widetilde{L}_{t+1}''$` | 11.0 | 2.4  |
|		`$L_t$`, `$L_{t-1}$`                         | 13.0                        | 4.9  | `$\widetilde{L}_{t}''$`, `$\widetilde{L}_{t+2}''$` | 10.7 | 2.8  |
|		`$\widetilde{L}_{t}''$`, `$\widetilde{L}_{t+1}''$` | 12.1                        | 2.3  | `$L_t$`, Solar                             | 10.0 | 2.9  |
|		`$L_t$`, `$W_t$`                             | 11.7                        | 3.0  | `$\widetilde{L}_{t}''$`, Temp                 | 9.5  | 3.4  |
|		`$\widetilde{L}_{t}''$`, `$\widetilde{L}_{t+2}''$` | 11.4                        | 2.9  | `$L_t$`, WindE                             | 9.3  | 2.5  |

]

---

# MLP Feature Importance

---

# Study Design and Evaluation

Rolling Window Forecasting Study:
- Length: 12 Months (10/2020 - 09/2021)
- 1-Month shifts
- Evaluation by RMSE

Competition Design:
- Only evaluating 09/2021
- Rank base on Score (relative RMSE)

`\begin{align}
	\text{Score} & = \text{RMSE}(\text{Model}) / \text{RMSE}(\textbf{naive}).
\end{align}`

]

Considered Models:

- .bold[GAM.full]
- .bold[GAM.red]
- .bold[DNN]
- .bold[naive]
- .bold[Combination of GAM.full, GAM.red, and DNN]

Two additional GAM models for diagnostic purposes:

- .bold[GAM.simple]
- .bold[GAM.no.Weather]

]

---

# Results

<div id="mhgdwmpptq" style="overflow-x:auto;overflow-y:auto;width:auto;height:auto;">
<style>html {
 font-family: 'LM Roman 10', -apple-system, BlinkMacSystemFont, 'Segoe UI', Roboto, Oxygen, Ubuntu, Cantarell, 'Helvetica Neue', 'Fira Sans', 'Droid Sans', Arial, sans-serif;
}

#mhgdwmpptq .gt_table {
  display: table;
  border-collapse: collapse;
  margin-left: auto;
  margin-right: auto;
  color: #333333;
  font-size: 16px;
  font-weight: normal;
  font-style: normal;
  background-color: #FFFFFF;
  width: auto;
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #A8A8A8;
  border-right-style: none;
  border-right-width: 2px;
  border-right-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #A8A8A8;
  border-left-style: none;
  border-left-width: 2px;
  border-left-color: #D3D3D3;
}

#mhgdwmpptq .gt_heading {
  background-color: #FFFFFF;
  text-align: center;
  border-bottom-color: #FFFFFF;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
}

#mhgdwmpptq .gt_title {
  color: #333333;
  font-size: 125%;
  font-weight: initial;
  padding-top: 4px;
  padding-bottom: 4px;
  padding-left: 5px;
  padding-right: 5px;
  border-bottom-color: #FFFFFF;
  border-bottom-width: 0;
}

#mhgdwmpptq .gt_subtitle {
  color: #333333;
  font-size: 85%;
  font-weight: initial;
  padding-top: 0;
  padding-bottom: 6px;
  padding-left: 5px;
  padding-right: 5px;
  border-top-color: #FFFFFF;
  border-top-width: 0;
}

#mhgdwmpptq .gt_bottom_border {
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
}

#mhgdwmpptq .gt_col_headings {
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
}

#mhgdwmpptq .gt_col_heading {
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: normal;
  text-transform: inherit;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
  vertical-align: bottom;
  padding-top: 5px;
  padding-bottom: 6px;
  padding-left: 5px;
  padding-right: 5px;
  overflow-x: hidden;
}

#mhgdwmpptq .gt_column_spanner_outer {
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: normal;
  text-transform: inherit;
  padding-top: 0;
  padding-bottom: 0;
  padding-left: 4px;
  padding-right: 4px;
}

#mhgdwmpptq .gt_column_spanner_outer:first-child {
  padding-left: 0;
}

#mhgdwmpptq .gt_column_spanner_outer:last-child {
  padding-right: 0;
}

#mhgdwmpptq .gt_column_spanner {
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  vertical-align: bottom;
  padding-top: 5px;
  padding-bottom: 5px;
  overflow-x: hidden;
  display: inline-block;
  width: 100%;
}

#mhgdwmpptq .gt_group_heading {
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: initial;
  text-transform: inherit;
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
  vertical-align: middle;
}

#mhgdwmpptq .gt_empty_group_heading {
  padding: 0.5px;
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: initial;
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  vertical-align: middle;
}

#mhgdwmpptq .gt_from_md > :first-child {
  margin-top: 0;
}

#mhgdwmpptq .gt_from_md > :last-child {
  margin-bottom: 0;
}

#mhgdwmpptq .gt_row {
  padding-top: 4px;
  padding-bottom: 4px;
  padding-left: 5px;
  padding-right: 5px;
  margin: 10px;
  border-top-style: solid;
  border-top-width: 1px;
  border-top-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
  vertical-align: middle;
  overflow-x: hidden;
}

#mhgdwmpptq .gt_stub {
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: initial;
  text-transform: inherit;
  border-right-style: solid;
  border-right-width: 2px;
  border-right-color: #D3D3D3;
  padding-left: 5px;
  padding-right: 5px;
}

#mhgdwmpptq .gt_stub_row_group {
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: initial;
  text-transform: inherit;
  border-right-style: solid;
  border-right-width: 2px;
  border-right-color: #D3D3D3;
  padding-left: 5px;
  padding-right: 5px;
  vertical-align: top;
}

#mhgdwmpptq .gt_row_group_first td {
  border-top-width: 2px;
}

#mhgdwmpptq .gt_summary_row {
  color: #333333;
  background-color: #FFFFFF;
  text-transform: inherit;
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
}

#mhgdwmpptq .gt_first_summary_row {
  border-top-style: solid;
  border-top-color: #D3D3D3;
}

#mhgdwmpptq .gt_first_summary_row.thick {
  border-top-width: 2px;
}

#mhgdwmpptq .gt_last_summary_row {
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
}

#mhgdwmpptq .gt_grand_summary_row {
  color: #333333;
  background-color: #FFFFFF;
  text-transform: inherit;
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
}

#mhgdwmpptq .gt_first_grand_summary_row {
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
  border-top-style: double;
  border-top-width: 6px;
  border-top-color: #D3D3D3;
}

#mhgdwmpptq .gt_striped {
  background-color: rgba(128, 128, 128, 0.05);
}

#mhgdwmpptq .gt_table_body {
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
}

#mhgdwmpptq .gt_footnotes {
  color: #333333;
  background-color: #FFFFFF;
  border-bottom-style: none;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 2px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 2px;
  border-right-color: #D3D3D3;
}

#mhgdwmpptq .gt_footnote {
  margin: 0px;
  font-size: 90%;
  padding-left: 4px;
  padding-right: 4px;
  padding-left: 5px;
  padding-right: 5px;
}

#mhgdwmpptq .gt_sourcenotes {
  color: #333333;
  background-color: #FFFFFF;
  border-bottom-style: none;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 2px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 2px;
  border-right-color: #D3D3D3;
}

#mhgdwmpptq .gt_sourcenote {
  font-size: 90%;
  padding-top: 4px;
  padding-bottom: 4px;
  padding-left: 5px;
  padding-right: 5px;
}

#mhgdwmpptq .gt_left {
  text-align: left;
}

#mhgdwmpptq .gt_center {
  text-align: center;
}

#mhgdwmpptq .gt_right {
  text-align: right;
  font-variant-numeric: tabular-nums;
}

#mhgdwmpptq .gt_font_normal {
  font-weight: normal;
}

#mhgdwmpptq .gt_font_bold {
  font-weight: bold;
}

#mhgdwmpptq .gt_font_italic {
  font-style: italic;
}

#mhgdwmpptq .gt_super {
  font-size: 65%;
}

#mhgdwmpptq .gt_two_val_uncert {
  display: inline-block;
  line-height: 1em;
  text-align: right;
  font-size: 60%;
  vertical-align: -0.25em;
  margin-left: 0.1em;
}

#mhgdwmpptq .gt_footnote_marks {
  font-style: italic;
  font-weight: normal;
  font-size: 75%;
  vertical-align: 0.4em;
}

#mhgdwmpptq .gt_asterisk {
  font-size: 100%;
  vertical-align: 0;
}

#mhgdwmpptq .gt_slash_mark {
  font-size: 0.7em;
  line-height: 0.7em;
  vertical-align: 0.15em;
}

#mhgdwmpptq .gt_fraction_numerator {
  font-size: 0.6em;
  line-height: 0.6em;
  vertical-align: 0.45em;
}

#mhgdwmpptq .gt_fraction_denominator {
 font-size: 0.6em;
 line-height: 0.6em;
 vertical-align: -0.05em;
}
</style>
<table class="gt_table" style="table-layout: fixed;">
 <colgroup>
 <col/>
 <col style="width:65px;"/>
 <col style="width:65px;"/>
 <col style="width:65px;"/>
 <col style="width:65px;"/>
 <col style="width:65px;"/>
 <col style="width:65px;"/>
 <col style="width:65px;"/>
 <col style="width:65px;"/>
 <col style="width:65px;"/>
 <col style="width:65px;"/>
 <col style="width:65px;"/>
 <col style="width:65px;"/>
 <col/>
 <col style="width:65px;"/>
 </colgroup>
 
 <thead class="gt_col_headings">
 <tr>
 <th class="gt_col_heading gt_columns_bottom_border gt_left" rowspan="1" colspan="1" style="font-weight: bold;">Overall</th>
 <th class="gt_col_heading gt_columns_bottom_border gt_center" rowspan="1" colspan="1" style="font-weight: bold;">20/10</th>
 <th class="gt_col_heading gt_columns_bottom_border gt_center" rowspan="1" colspan="1" style="font-weight: bold;">20/11</th>
 <th class="gt_col_heading gt_columns_bottom_border gt_center" rowspan="1" colspan="1" style="font-weight: bold;">20/12</th>
 <th class="gt_col_heading gt_columns_bottom_border gt_center" rowspan="1" colspan="1" style="font-weight: bold;">21/1</th>
 <th class="gt_col_heading gt_columns_bottom_border gt_center" rowspan="1" colspan="1" style="font-weight: bold;">21/2</th>
 <th class="gt_col_heading gt_columns_bottom_border gt_center" rowspan="1" colspan="1" style="font-weight: bold;">21/3</th>
 <th class="gt_col_heading gt_columns_bottom_border gt_center" rowspan="1" colspan="1" style="font-weight: bold;">21/4</th>
 <th class="gt_col_heading gt_columns_bottom_border gt_center" rowspan="1" colspan="1" style="font-weight: bold;">21/5</th>
 <th class="gt_col_heading gt_columns_bottom_border gt_center" rowspan="1" colspan="1" style="font-weight: bold;">21/6</th>
 <th class="gt_col_heading gt_columns_bottom_border gt_center" rowspan="1" colspan="1" style="font-weight: bold;">21/7</th>
 <th class="gt_col_heading gt_columns_bottom_border gt_center" rowspan="1" colspan="1" style="font-weight: bold;">21/8</th>
 <th class="gt_col_heading gt_columns_bottom_border gt_center" rowspan="1" colspan="1" style="font-weight: bold;">21/9</th>
 <th class="gt_col_heading gt_columns_bottom_border gt_center" rowspan="1" colspan="1" style="font-weight: bold;"> </th>
 <th class="gt_col_heading gt_columns_bottom_border gt_center" rowspan="1" colspan="1" style="font-weight: bold;">Avg</th>
 </tr>
 </thead>
 <tbody class="gt_table_body">
 <tr><td class="gt_row gt_right gt_stub" style="font-weight: bold;">GAM.full</td>
<td class="gt_row gt_center" style="background-color: #88C567; color: #000000;">.1236 (.436)</td>
<td class="gt_row gt_center" style="background-color: #9BCC65; color: #000000;">.0705 (.416)</td>
<td class="gt_row gt_center" style="background-color: #90C866; color: #000000;">.0499 (.436)</td>
<td class="gt_row gt_center" style="background-color: #83C368; color: #000000;">.0533 (.425)</td>
<td class="gt_row gt_center" style="background-color: #7DC168; color: #000000;">.0987 (.445)</td>
<td class="gt_row gt_center" style="background-color: #79C068; color: #000000;">.1203 (.423)</td>
<td class="gt_row gt_center" style="background-color: #6FBD69; color: #000000;">.1449 (.429)</td>
<td class="gt_row gt_center" style="background-color: #E6E55C; color: #000000;">.1747 (.434)</td>
<td class="gt_row gt_center" style="background-color: #6EBD69; color: #000000;">.1368 (.437)</td>
<td class="gt_row gt_center" style="background-color: #94CA66; color: #000000;">.1536 (.461)</td>
<td class="gt_row gt_center" style="background-color: #7AC068; color: #000000;">.1383 (.398)</td>
<td class="gt_row gt_center" style="background-color: #6BBB6A; color: #000000;">.1164 (.431)</td>
<td class="gt_row gt_center" style="border-left-width: 1px; border-left-style: solid; border-left-color: transparent; border-right-width: 1px; border-right-style: solid; border-right-color: transparent; border-top-width: 1px; border-top-style: solid; border-top-color: transparent; border-bottom-width: 1px; border-bottom-style: solid; border-bottom-color: transparent;"></td>
<td class="gt_row gt_center" style="background-color: #88C567; color: #000000;">.1151 (.431)</td></tr>
 <tr><td class="gt_row gt_right gt_stub" style="font-weight: bold;">GAM.red</td>
<td class="gt_row gt_center" style="background-color: #92C966; color: #000000;">.1241 (.438)</td>
<td class="gt_row gt_center" style="background-color: #8CC667; color: #000000;">.0700 (.413)</td>
<td class="gt_row gt_center" style="background-color: #7CC168; color: #000000;">.0495 (.433)</td>
<td class="gt_row gt_center" style="background-color: #90C866; color: #000000;">.0536 (.427)</td>
<td class="gt_row gt_center" style="background-color: #8DC767; color: #000000;">.0993 (.448)</td>
<td class="gt_row gt_center" style="background-color: #77C069; color: #000000;">.1202 (.422)</td>
<td class="gt_row gt_center" style="background-color: #8FC766; color: #000000;">.1467 (.434)</td>
<td class="gt_row gt_center" style="background-color: #EAE65B; color: #000000;">.1750 (.434)</td>
<td class="gt_row gt_center" style="background-color: #8FC866; color: #000000;">.1385 (.442)</td>
<td class="gt_row gt_center" style="background-color: #B4D462; color: #000000;">.1554 (.466)</td>
<td class="gt_row gt_center" style="background-color: #6BBC6A; color: #000000;">.1374 (.395)</td>
<td class="gt_row gt_center" style="background-color: #66BA6A; color: #000000;">.1162 (.431)</td>
<td class="gt_row gt_center" style="border-left-width: 1px; border-left-style: solid; border-left-color: transparent; border-right-width: 1px; border-right-style: solid; border-right-color: transparent; border-top-width: 1px; border-top-style: solid; border-top-color: transparent; border-bottom-width: 1px; border-bottom-style: solid; border-bottom-color: transparent;"></td>
<td class="gt_row gt_center" style="background-color: #90C866; color: #000000;">.1155 (.433)</td></tr>
 <tr><td class="gt_row gt_right gt_stub" style="font-weight: bold;">DNN</td>
<td class="gt_row gt_center" style="background-color: #7CC168; color: #000000;">.1230 (.434)</td>
<td class="gt_row gt_center" style="background-color: #98CB65; color: #000000;">.0704 (.416)</td>
<td class="gt_row gt_center" style="background-color: #B7D562; color: #000000;">.0507 (.443)</td>
<td class="gt_row gt_center" style="background-color: #B8D562; color: #000000;">.0545 (.434)</td>
<td class="gt_row gt_center" style="background-color: #E4E45C; color: #000000;">.1027 (.463)</td>
<td class="gt_row gt_center" style="background-color: #D7E05E; color: #000000;">.1251 (.439)</td>
<td class="gt_row gt_center" style="background-color: #DCE15D; color: #000000;">.1514 (.448)</td>
<td class="gt_row gt_center" style="background-color: #66BA6A; color: #000000;">.1653 (.410)</td>
<td class="gt_row gt_center" style="background-color: #FFE54E; color: #000000;">.1474 (.471)</td>
<td class="gt_row gt_center" style="background-color: #98CB65; color: #000000;">.1538 (.461)</td>
<td class="gt_row gt_center" style="background-color: #B6D562; color: #000000;">.1422 (.409)</td>
<td class="gt_row gt_center" style="background-color: #EBE65B; color: #000000;">.1225 (.454)</td>
<td class="gt_row gt_center" style="border-left-width: 1px; border-left-style: solid; border-left-color: transparent; border-right-width: 1px; border-right-style: solid; border-right-color: transparent; border-top-width: 1px; border-top-style: solid; border-top-color: transparent; border-bottom-width: 1px; border-bottom-style: solid; border-bottom-color: transparent;"></td>
<td class="gt_row gt_center" style="background-color: #B8D562; color: #000000;">.1174 (.440)</td></tr>
 <tr><td class="gt_row gt_right gt_stub" style="font-weight: bold;">Combination</td>
<td class="gt_row gt_center" style="background-color: #66BA6A; color: #000000;">.1220 (.431)</td>
<td class="gt_row gt_center" style="background-color: #66BA6A; color: #000000;">.0689 (.407)</td>
<td class="gt_row gt_center" style="background-color: #66BA6A; color: #000000;">.0491 (.429)</td>
<td class="gt_row gt_center" style="background-color: #66BA6A; color: #000000;">.0527 (.420)</td>
<td class="gt_row gt_center" style="background-color: #66BA6A; color: #000000;">.0979 (.442)</td>
<td class="gt_row gt_center" style="background-color: #66BA6A; color: #000000;">.1194 (.419)</td>
<td class="gt_row gt_center" style="background-color: #66BA6A; color: #000000;">.1444 (.427)</td>
<td class="gt_row gt_center" style="background-color: #92C966; color: #000000;">.1684 (.418)</td>
<td class="gt_row gt_center" style="background-color: #66BA6A; color: #000000;">.1364 (.436)</td>
<td class="gt_row gt_center" style="background-color: #66BA6A; color: #000000;">.1511 (.453)</td>
<td class="gt_row gt_center" style="background-color: #66BA6A; color: #000000;">.1371 (.394)</td>
<td class="gt_row gt_center" style="background-color: #6BBB6A; color: #000000;">.1164 (.431)</td>
<td class="gt_row gt_center" style="border-left-width: 1px; border-left-style: solid; border-left-color: transparent; border-right-width: 1px; border-right-style: solid; border-right-color: transparent; border-top-width: 1px; border-top-style: solid; border-top-color: transparent; border-bottom-width: 1px; border-bottom-style: solid; border-bottom-color: transparent;"></td>
<td class="gt_row gt_center" style="background-color: #66BA6A; color: #000000;">.1136 (.425)</td></tr>
 <tr><td class="gt_row gt_right gt_stub" style="font-weight: bold;">GAM.noWeather</td>
<td class="gt_row gt_center" style="background-color: #FFEC56; color: #000000;">.1301 (.459)</td>
<td class="gt_row gt_center" style="background-color: #B2D363; color: #000000;">.0712 (.421)</td>
<td class="gt_row gt_center" style="background-color: #C1D861; color: #000000;">.0509 (.445)</td>
<td class="gt_row gt_center" style="background-color: #7AC068; color: #000000;">.0531 (.423)</td>
<td class="gt_row gt_center" style="background-color: #FFE34B; color: #000000;">.1061 (.479)</td>
<td class="gt_row gt_center" style="background-color: #DAE15E; color: #000000;">.1253 (.440)</td>
<td class="gt_row gt_center" style="background-color: #95CA66; color: #000000;">.1471 (.435)</td>
<td class="gt_row gt_center" style="background-color: #FFE048; color: #000000;">.1822 (.452)</td>
<td class="gt_row gt_center" style="background-color: #ACD163; color: #000000;">.1401 (.447)</td>
<td class="gt_row gt_center" style="background-color: #E8E55C; color: #000000;">.1584 (.475)</td>
<td class="gt_row gt_center" style="background-color: #B9D662; color: #000000;">.1424 (.409)</td>
<td class="gt_row gt_center" style="background-color: #FFE852; color: #000000;">.1249 (.463)</td>
<td class="gt_row gt_center" style="border-left-width: 1px; border-left-style: solid; border-left-color: transparent; border-right-width: 1px; border-right-style: solid; border-right-color: transparent; border-top-width: 1px; border-top-style: solid; border-top-color: transparent; border-bottom-width: 1px; border-bottom-style: solid; border-bottom-color: transparent;"></td>
<td class="gt_row gt_center" style="background-color: #DFE25D; color: #000000;">.1193 (.447)</td></tr>
 <tr><td class="gt_row gt_right gt_stub" style="font-weight: bold;">GAM.simple</td>
<td class="gt_row gt_center" style="background-color: #FFB326; color: #000000;">.1531 (.540)</td>
<td class="gt_row gt_center" style="background-color: #FFA428; color: #000000;">.0940 (.555)</td>
<td class="gt_row gt_center" style="background-color: #FFC127; color: #000000;">.0584 (.510)</td>
<td class="gt_row gt_center" style="background-color: #FFB126; color: #000000;">.0672 (.535)</td>
<td class="gt_row gt_center" style="background-color: #FFA826; color: #000000;">.1267 (.571)</td>
<td class="gt_row gt_center" style="background-color: #FFB526; color: #000000;">.1500 (.527)</td>
<td class="gt_row gt_center" style="background-color: #FFB626; color: #000000;">.1797 (.532)</td>
<td class="gt_row gt_center" style="background-color: #FFB526; color: #000000;">.2093 (.519)</td>
<td class="gt_row gt_center" style="background-color: #FFAE26; color: #000000;">.1733 (.553)</td>
<td class="gt_row gt_center" style="background-color: #FFB126; color: #000000;">.1873 (.562)</td>
<td class="gt_row gt_center" style="background-color: #FFC127; color: #000000;">.1671 (.480)</td>
<td class="gt_row gt_center" style="background-color: #FFBB27; color: #000000;">.1413 (.524)</td>
<td class="gt_row gt_center" style="border-left-width: 1px; border-left-style: solid; border-left-color: transparent; border-right-width: 1px; border-right-style: solid; border-right-color: transparent; border-top-width: 1px; border-top-style: solid; border-top-color: transparent; border-bottom-width: 1px; border-bottom-style: solid; border-bottom-color: transparent;"></td>
<td class="gt_row gt_center" style="background-color: #FFB526; color: #000000;">.1423 (.533)</td></tr>
 <tr><td class="gt_row gt_right gt_stub" style="font-weight: bold;">Naive</td>
<td class="gt_row gt_center" style="background-color: #EE5250; color: #000000;">.2833</td>
<td class="gt_row gt_center" style="background-color: #EE5250; color: #000000;">.1693</td>
<td class="gt_row gt_center" style="background-color: #EE5250; color: #000000;">.1144</td>
<td class="gt_row gt_center" style="background-color: #EE5250; color: #000000;">.1255</td>
<td class="gt_row gt_center" style="background-color: #EE5250; color: #000000;">.2217</td>
<td class="gt_row gt_center" style="background-color: #EE5250; color: #000000;">.2847</td>
<td class="gt_row gt_center" style="background-color: #EE5250; color: #000000;">.3380</td>
<td class="gt_row gt_center" style="background-color: #EE5250; color: #000000;">.4029</td>
<td class="gt_row gt_center" style="background-color: #EE5250; color: #000000;">.3131</td>
<td class="gt_row gt_center" style="background-color: #EE5250; color: #000000;">.3334</td>
<td class="gt_row gt_center" style="background-color: #EE5250; color: #000000;">.3478</td>
<td class="gt_row gt_center" style="background-color: #EE5250; color: #000000;">.2699</td>
<td class="gt_row gt_center" style="border-left-width: 1px; border-left-style: solid; border-left-color: transparent; border-right-width: 1px; border-right-style: solid; border-right-color: transparent; border-top-width: 1px; border-top-style: solid; border-top-color: transparent; border-bottom-width: 1px; border-bottom-style: solid; border-bottom-color: transparent;"></td>
<td class="gt_row gt_center" style="background-color: #EE5250; color: #000000;">.2670</td></tr>
 </tbody>
 
 
</table>
</div>

---

# Wrap-Up

Estimating high-resolution elecricity peak demand using lower-resolution data:

- .bold[GAM.full] and .bold[GAM.red] perform similar
- .bold[DNN] beats .bold[GAM.full] in some Months
- .bold[Combination of GAM.full, GAM.red, and DNN] performs best

- Weather variables improve the skill score by 1.5 percentage points on average
- .bold[DNN] performs better at predicting maximum peak loads

]

The proposed model performed best in the competition.

- Outperforming place 2 by 1 percentage point (43.6% vs. 42.6%)
- We used a slightly different model in the competition (including selected 3-way interactions)

]
]

<a href="https://github.com/BerriJ" class="github-corner" aria-label="View source on Github"><svg width="80" height="80" viewBox="0 0 250 250" style="fill:#f2f2f2; color:#212121; position: absolute; top: 0; border: 0; right: 0;" aria-hidden="true"><path d="M0,0 L115,115 L130,115 L142,142 L250,250 L250,0 Z"></path><path d="M128.3,109.0 C113.8,99.7 119.0,89.6 119.0,89.6 C122.0,82.7 120.5,78.6 120.5,78.6 C119.2,72.0 123.4,76.3 123.4,76.3 C127.3,80.9 125.5,87.3 125.5,87.3 C122.9,97.6 130.6,101.9 134.4,103.2" fill="currentColor" style="transform-origin: 130px 106px;" class="octo-arm"></path><path d="M115.0,115.0 C114.9,115.1 118.7,116.5 119.8,115.4 L133.7,101.6 C136.9,99.2 139.9,98.4 142.2,98.6 C133.8,88.0 127.5,74.4 143.8,58.0 C148.5,53.4 154.0,51.2 159.7,51.0 C160.3,49.4 163.2,43.6 171.4,40.1 C171.4,40.1 176.1,42.5 178.8,56.2 C183.1,58.6 187.2,61.8 190.9,65.4 C194.5,69.0 197.7,73.2 200.1,77.6 C213.8,80.2 216.3,84.9 216.3,84.9 C212.7,93.1 206.9,96.0 205.4,96.6 C205.1,102.4 203.0,107.8 198.3,112.5 C181.9,128.9 168.3,122.5 157.7,114.1 C157.9,116.9 156.7,120.9 152.7,124.9 L141.0,136.5 C139.8,137.7 141.6,141.9 141.8,141.8 Z" fill="currentColor" class="octo-body"></path></svg></a><style>.github-corner:hover .octo-arm{animation:octocat-wave 560ms ease-in-out}@keyframes octocat-wave{0%,100%{transform:rotate(0)}20%,60%{transform:rotate(-25deg)}40%,80%{transform:rotate(10deg)}}@media (max-width:500px){.github-corner:hover .octo-arm{animation:none}.github-corner .octo-arm{animation:octocat-wave 560ms ease-in-out}}</style>

---
name:references

# References 1

<cite><a id='bib-akiba2019optuna'></a><a href="#cite-akiba2019optuna">Akiba, T., S. Sano, T. Yanase, et al.</a>
(2019).
&ldquo;Optuna: A next-generation hyperparameter optimization framework&rdquo;.
In: 
Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery &amp; data mining.
, pp. 2623&ndash;2631.</cite>

<cite><a id='bib-berrisch2022high'></a><a href="#cite-berrisch2022high">Berrisch, J., M. Narajewski, and F. Ziel</a>
(2022).
&ldquo;High-Resolution Peak Demand Estimation Using Generalized Additive Models and Deep Neural Networks&rdquo;.
In: arXiv preprint arXiv:2203.03342.</cite>

<cite><a id='bib-szekely2007measuring'></a><a href="#cite-szekely2007measuring">Székely, G. J., M. L. Rizzo, and N. K. Bakirov</a>
(2007).
&ldquo;Measuring and testing dependence by correlation of distances&rdquo;.
In: The annals of statistics 35.6, pp. 2769&ndash;2794.</cite>