High-Resolution Peak Demand Estimation Using Generalized Additive Models and Deep Neural Networks

.title[
# High-Resolution Peak Demand Estimation Using Generalized Additive Models and Deep Neural Networks
]
.subtitle[
## Jonathan Berrisch, Michał Narajewski, Florian Ziel
]
.author[
### University of Duisburg-Essen
]
.date[
### 2023-09-13
]

---

# Motivation

- Organized by Western Power Distribution and Catapult Energy Systems
- Does limited high-resolution monitoring help estimate future high-resolution peak loads?

The Objective:
- Estimate minimum and maximum electricity load values (one-minute resolution)
- Given data with only a 30-minute resolution
- One single substation, every half-hour of September 2021

Data:
- From Nov. 2019 to Sept. 2021 (30-minute resolution)
- MERRA-2 weather reanalysis data from five locations close to the substation

---

# Motivation

### Location Overview

---
  
# Motivation

### Data Overview

---

# Data

Targets: `$\Delta^{\min}_t$` and `$\Delta^{\max}_t$`

Possible explanatory variables:

The *half-hourly* load: `$L_t$`

Discrete second order central difference (DSOCD):

`$L_t'' = L_{t-1}-2L_t+L_{t+1}$`

Deterministic components (to capture potential seasonal characteristics)
- Daily `$D_t$` number of hours in a day
- Weekly `$W_t$` number of hours in a week
- Annual  `$A_t$` number of hours in a meteorological year with `$365.24$` days
]

Weather Inputs: Temperature, Windspeed (North / East), Solar, Humidity

The Figure (next slide) shows how these variables correlate.
- Lower triangle:
  - Pearson's correlation
- Upper triangle:
  - Distance correlation

Distance correlation: non-linear dependency measure that takes values in `$[0, 1]$` and characterizes stochastic independence <a id='cite-szekely2007measuring'></a><a href='#bib-szekely2007measuring'>Székely, Rizzo, and Bakirov (2007)</a>.

]

---

# Correlation

---

# Data

---

# Feature Set and Models

Selected features based on preliminary analysis:

| Variable type  | Included feature  | Number  |
|---|---|---|
| Lagged load  | `$L_{t-1}$`, `$L_t$`, `$L_{t+1}$`  |  3 |
| Lagged DSOCD load  |  `$\widetilde{L}_{t-4}'',\ldots , \widetilde{L}_{t+4}''$` | 9  |
| Weather inputs  | Temp, Solar, WindN, WindE, Press, Humid  | 6  |
| Seasonal inputs  |  `$D_t$`, `$W_t$`, `$A_t$` | 3  |

]

Considered Models:

- Generalized Additive Model (GAM)
- Multilayer Perceptron Network (MLP)
- Combinations of the above

Competition Benchmark:

`$$L_t^{\max} = L_t$$`
`$$L_t^{\min} = L_t$$`

]

---

# Modelling Approach: GAM

### Generalized Additive Model (GAM)

`\begin{equation}
	\Delta^{m}_t = \sum_{i=1}^L f_i( X_{t,1}, \ldots, X_{t,N}) + \epsilon_t
	\label{eq_GAM_gen}
\end{equation}`
where `$m\in\{\min , \max\}$`

Traditional framework using cubic B-splines

*GAM.full* Specification (with all 2-way interactions):

`\begin{align}
	\Delta^{m}_t =& \sum_{i=1}^N b_{k_0}( X_{t,i} ) + \nonumber \\
  &\sum_{i=1}^N \sum_{j=1,j>i}^N b_{k_1,k_2}( X_{t,i}, X_{t,j} ) +\epsilon_t
	\label{eq_big_GAM}
\end{align}`

]

`$b_{k_0}$` and `$b_{k_1,k_2}$` denote univariate and bivariate splines with `$k_0$`, and `$(k_1,k_2)$` knots.
Tensor interaction splines -> only capture the joint effects.

We set `$k_0=27$` and `$k_1=k_2=9$`. Thus, linear terms are specified by `$27$` parameters and bivariate terms by `$81$` parameters.

*GAM.red*:

Interactions with `$L_t$`, `$\widetilde{L}_t''$` and `$\text{Solar}_t$`.

*GAM.simple*:
`\begin{align}
	\Delta^{m}_t =  b_{k_0}( L_t ) +  b_{k_0}( \widetilde{L}_t'') + b_{k_0}( \text{Solar}_t )
	+\epsilon
	\label{eq_simple_GAM}
\end{align}`

]

---

# Modelling Approach: MLP

Multilayer Perceptron Network:
<img src="fig/nn_fig.svg" width="500px" style="display: block; margin: auto;" />

Hyperparameters are tuned using OPTUNA Python package <a id='cite-akiba2019optuna'></a><a href='#bib-akiba2019optuna'>Akiba, Sano, Yanase, Ohta, and Koyama (2019)</a>

]

.pull-right[
- Input feature selection
- Number of hidden layers -- either 2 or 3
- Dropout layer -- whether to use it after the input layer and, if yes, at what rate.
- Activation functions in the hidden layers: elu, relu, sigmoid, softmax, softplus, and tanh
- Number of neurons in the hidden layer drawn on an exp-scale from `$[4,128]$`
- `$L_1$` regularization -- whether to use it on the hidden layers and, if yes, at what rate.
- Learning rate for the Adam optimization algorithm drawn on an exp-scale from `$\left(10^{-5}, 10^{-1}\right)$` interval
]

---

# GAM Parameter Significance

|		Min                             | EDF                         | F    | Max                             | EDF  | F    |
| --- |
|		`$L_t$`                                    | 9.5                         | 8.9  | `$L_t$`                                    | 10.8 | 17.6 |
|		`$\widetilde{L}_{t-1}''$`                     | 8.8                         | 4.4  | `$\widetilde{L}_{t-2}''$`                     | 6.7  | 4.9  |
|		`$\widetilde{L}_{t+1}''$`                     | 8.4                         | 4.8  | `$L_{t-1}$`                                | 6.3  | 6.9  |
|		`$\widetilde{L}_{t}''$`                       | 8.2                         | 4.6  | `$L_{t+1}$`                                | 5.4  | 5.0  |
|		Humid                                    | 6.1                         | 3.7  | `$D_t$`                                    | 4.8  | 3.1  |
|		WindE                                    | 5.1                         | 8.9  | Temp                                     | 4.1  | 11.5 |
|		`$A_t$`                                    | 4.7                         | 4.9  | Solar                                    | 4.0  | 5.5  |
|		WindN                                    | 4.2                         | 8.5  | WindE                                    | 3.8  | 5.8  |
|		Temp                                     | 3.5                         | 3.6  | `$\widetilde{L}_{t+4}''$`                     | 3.2  | 2.1  |
|		`$L_{t-1}$`                                | 3.3                         | 1.3  | `$\widetilde{L}_{t+1}''$`                     | 3.1  | 2.2  |

]

|		Min                             | EDF                         | F    | Max                             | EDF  | F    |
| --- |
|		`$L_t$`, `$A_t$`                             | 26.7                        | 7.8  | `$L_t$`, `$A_t$`                             | 37.8 | 28.1 |
|		`$L_t$`, Solar                             | 20.7                        | 11.2 | `$L_t$`, `$D_t$`                             | 25.1 | 20.6 |
|		`$L_t$`, `$D_t$`                             | 18.3                        | 8.2  | Solar, `$A_t$`                             | 15.4 | 6.4  |
|		`$L_t$`, `$L_{t+1}$`                         | 17.1                        | 4.0  | `$\widetilde{L}_{t}''$`, `$\widetilde{L}_{t-1}''$` | 14.9 | 6.5  |
|		`$\widetilde{L}_{t}''$`, `$\widetilde{L}_{t-1}''$` | 13.5                        | 2.5  | Solar, `$D_t$`                             | 11.0 | 8.5  |
|		`$L_t$`, Temp                              | 13.3                        | 3.1  | `$\widetilde{L}_{t}''$`, `$\widetilde{L}_{t+1}''$` | 11.0 | 2.4  |
|		`$L_t$`, `$L_{t-1}$`                         | 13.0                        | 4.9  | `$\widetilde{L}_{t}''$`, `$\widetilde{L}_{t+2}''$` | 10.7 | 2.8  |
|		`$\widetilde{L}_{t}''$`, `$\widetilde{L}_{t+1}''$` | 12.1                        | 2.3  | `$L_t$`, Solar                             | 10.0 | 2.9  |
|		`$L_t$`, `$W_t$`                             | 11.7                        | 3.0  | `$\widetilde{L}_{t}''$`, Temp                 | 9.5  | 3.4  |
|		`$\widetilde{L}_{t}''$`, `$\widetilde{L}_{t+2}''$` | 11.4                        | 2.9  | `$L_t$`, WindE                             | 9.3  | 2.5  |

]

---

# MLP Feature Importance

---

# Study Design and Evaluation

Rolling Window Forecasting Study:
- Length: 12 Months (10/2020 - 09/2021)
- 1-Month shifts
- Evaluation by RMSE

Competition Design:
- Only evaluating 09/2021
- Rank base on Score (relative RMSE)

`\begin{align}
	\text{Score} & = \text{RMSE}(\text{Model}) / \text{RMSE}(\textbf{naive}).
\end{align}`

]

Considered Models:

- .bold[GAM.full]
- .bold[GAM.red]
- .bold[DNN]
- .bold[naive]
- .bold[Combination of GAM.full, GAM.red, and DNN]

Two additional GAM models for diagnostic purposes:

- .bold[GAM.simple]
- .bold[GAM.no.Weather]

]

---

# Results

<div id="bcatjjdkab" style="padding-left:0px;padding-right:0px;padding-top:10px;padding-bottom:10px;overflow-x:auto;overflow-y:auto;width:auto;height:auto;">
<style>#bcatjjdkab table {
 font-family: 'LM Roman 10', system-ui, 'Segoe UI', Roboto, Helvetica, Arial, sans-serif, 'Apple Color Emoji', 'Segoe UI Emoji', 'Segoe UI Symbol', 'Noto Color Emoji';
 -webkit-font-smoothing: antialiased;
 -moz-osx-font-smoothing: grayscale;
}

#bcatjjdkab thead, #bcatjjdkab tbody, #bcatjjdkab tfoot, #bcatjjdkab tr, #bcatjjdkab td, #bcatjjdkab th {
  border-style: none;
}

#bcatjjdkab p {
  margin: 0;
  padding: 0;
}

#bcatjjdkab .gt_table {
  display: table;
  border-collapse: collapse;
  line-height: normal;
  margin-left: auto;
  margin-right: auto;
  color: #333333;
  font-size: 16px;
  font-weight: normal;
  font-style: normal;
  background-color: #FFFFFF;
  width: auto;
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #A8A8A8;
  border-right-style: none;
  border-right-width: 2px;
  border-right-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #A8A8A8;
  border-left-style: none;
  border-left-width: 2px;
  border-left-color: #D3D3D3;
}

#bcatjjdkab .gt_caption {
  padding-top: 4px;
  padding-bottom: 4px;
}

#bcatjjdkab .gt_title {
  color: #333333;
  font-size: 125%;
  font-weight: initial;
  padding-top: 4px;
  padding-bottom: 4px;
  padding-left: 5px;
  padding-right: 5px;
  border-bottom-color: #FFFFFF;
  border-bottom-width: 0;
}

#bcatjjdkab .gt_subtitle {
  color: #333333;
  font-size: 85%;
  font-weight: initial;
  padding-top: 3px;
  padding-bottom: 5px;
  padding-left: 5px;
  padding-right: 5px;
  border-top-color: #FFFFFF;
  border-top-width: 0;
}

#bcatjjdkab .gt_heading {
  background-color: #FFFFFF;
  text-align: center;
  border-bottom-color: #FFFFFF;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
}

#bcatjjdkab .gt_bottom_border {
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
}

#bcatjjdkab .gt_col_headings {
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
}

#bcatjjdkab .gt_col_heading {
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: normal;
  text-transform: inherit;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
  vertical-align: bottom;
  padding-top: 5px;
  padding-bottom: 6px;
  padding-left: 5px;
  padding-right: 5px;
  overflow-x: hidden;
}

#bcatjjdkab .gt_column_spanner_outer {
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: normal;
  text-transform: inherit;
  padding-top: 0;
  padding-bottom: 0;
  padding-left: 4px;
  padding-right: 4px;
}

#bcatjjdkab .gt_column_spanner_outer:first-child {
  padding-left: 0;
}

#bcatjjdkab .gt_column_spanner_outer:last-child {
  padding-right: 0;
}

#bcatjjdkab .gt_column_spanner {
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  vertical-align: bottom;
  padding-top: 5px;
  padding-bottom: 5px;
  overflow-x: hidden;
  display: inline-block;
  width: 100%;
}

#bcatjjdkab .gt_spanner_row {
  border-bottom-style: hidden;
}

#bcatjjdkab .gt_group_heading {
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: initial;
  text-transform: inherit;
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
  vertical-align: middle;
  text-align: left;
}

#bcatjjdkab .gt_empty_group_heading {
  padding: 0.5px;
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: initial;
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  vertical-align: middle;
}

#bcatjjdkab .gt_from_md > :first-child {
  margin-top: 0;
}

#bcatjjdkab .gt_from_md > :last-child {
  margin-bottom: 0;
}

#bcatjjdkab .gt_row {
  padding-top: 4px;
  padding-bottom: 4px;
  padding-left: 5px;
  padding-right: 5px;
  margin: 10px;
  border-top-style: solid;
  border-top-width: 1px;
  border-top-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 1px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 1px;
  border-right-color: #D3D3D3;
  vertical-align: middle;
  overflow-x: hidden;
}

#bcatjjdkab .gt_stub {
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: initial;
  text-transform: inherit;
  border-right-style: solid;
  border-right-width: 2px;
  border-right-color: #D3D3D3;
  padding-left: 5px;
  padding-right: 5px;
}

#bcatjjdkab .gt_stub_row_group {
  color: #333333;
  background-color: #FFFFFF;
  font-size: 100%;
  font-weight: initial;
  text-transform: inherit;
  border-right-style: solid;
  border-right-width: 2px;
  border-right-color: #D3D3D3;
  padding-left: 5px;
  padding-right: 5px;
  vertical-align: top;
}

#bcatjjdkab .gt_row_group_first td {
  border-top-width: 2px;
}

#bcatjjdkab .gt_row_group_first th {
  border-top-width: 2px;
}

#bcatjjdkab .gt_summary_row {
  color: #333333;
  background-color: #FFFFFF;
  text-transform: inherit;
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
}

#bcatjjdkab .gt_first_summary_row {
  border-top-style: solid;
  border-top-color: #D3D3D3;
}

#bcatjjdkab .gt_first_summary_row.thick {
  border-top-width: 2px;
}

#bcatjjdkab .gt_last_summary_row {
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
}

#bcatjjdkab .gt_grand_summary_row {
  color: #333333;
  background-color: #FFFFFF;
  text-transform: inherit;
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
}

#bcatjjdkab .gt_first_grand_summary_row {
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
  border-top-style: double;
  border-top-width: 6px;
  border-top-color: #D3D3D3;
}

#bcatjjdkab .gt_last_grand_summary_row_top {
  padding-top: 8px;
  padding-bottom: 8px;
  padding-left: 5px;
  padding-right: 5px;
  border-bottom-style: double;
  border-bottom-width: 6px;
  border-bottom-color: #D3D3D3;
}

#bcatjjdkab .gt_striped {
  background-color: rgba(128, 128, 128, 0.05);
}

#bcatjjdkab .gt_table_body {
  border-top-style: solid;
  border-top-width: 2px;
  border-top-color: #D3D3D3;
  border-bottom-style: solid;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
}

#bcatjjdkab .gt_footnotes {
  color: #333333;
  background-color: #FFFFFF;
  border-bottom-style: none;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 2px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 2px;
  border-right-color: #D3D3D3;
}

#bcatjjdkab .gt_footnote {
  margin: 0px;
  font-size: 90%;
  padding-top: 4px;
  padding-bottom: 4px;
  padding-left: 5px;
  padding-right: 5px;
}

#bcatjjdkab .gt_sourcenotes {
  color: #333333;
  background-color: #FFFFFF;
  border-bottom-style: none;
  border-bottom-width: 2px;
  border-bottom-color: #D3D3D3;
  border-left-style: none;
  border-left-width: 2px;
  border-left-color: #D3D3D3;
  border-right-style: none;
  border-right-width: 2px;
  border-right-color: #D3D3D3;
}

#bcatjjdkab .gt_sourcenote {
  font-size: 90%;
  padding-top: 4px;
  padding-bottom: 4px;
  padding-left: 5px;
  padding-right: 5px;
}

#bcatjjdkab .gt_left {
  text-align: left;
}

#bcatjjdkab .gt_center {
  text-align: center;
}

#bcatjjdkab .gt_right {
  text-align: right;
  font-variant-numeric: tabular-nums;
}

#bcatjjdkab .gt_font_normal {
  font-weight: normal;
}

#bcatjjdkab .gt_font_bold {
  font-weight: bold;
}

#bcatjjdkab .gt_font_italic {
  font-style: italic;
}

#bcatjjdkab .gt_super {
  font-size: 65%;
}

#bcatjjdkab .gt_footnote_marks {
  font-size: 75%;
  vertical-align: 0.4em;
  position: initial;
}

#bcatjjdkab .gt_asterisk {
  font-size: 100%;
  vertical-align: 0;
}

#bcatjjdkab .gt_indent_1 {
  text-indent: 5px;
}

#bcatjjdkab .gt_indent_2 {
  text-indent: 10px;
}

#bcatjjdkab .gt_indent_3 {
  text-indent: 15px;
}

#bcatjjdkab .gt_indent_4 {
  text-indent: 20px;
}

#bcatjjdkab .gt_indent_5 {
 text-indent: 25px;
}
</style>
<table class="gt_table" style="table-layout: fixed;" data-quarto-disable-processing="false" data-quarto-bootstrap="false">
 <colgroup>
 <col/>
 <col style="width:65px;"/>
 <col style="width:65px;"/>
 <col style="width:65px;"/>
 <col style="width:65px;"/>
 <col style="width:65px;"/>
 <col style="width:65px;"/>
 <col style="width:65px;"/>
 <col style="width:65px;"/>
 <col style="width:65px;"/>
 <col style="width:65px;"/>
 <col style="width:65px;"/>
 <col style="width:65px;"/>
 <col/>
 <col style="width:65px;"/>
 </colgroup>
 <thead>
 
 <tr class="gt_col_headings">
 <th class="gt_col_heading gt_columns_bottom_border gt_left" rowspan="1" colspan="1" style="font-weight: bold;" scope="col" id="Overall">Overall</th>
 <th class="gt_col_heading gt_columns_bottom_border gt_center" rowspan="1" colspan="1" style="font-weight: bold;" scope="col" id="20/10">20/10</th>
 <th class="gt_col_heading gt_columns_bottom_border gt_center" rowspan="1" colspan="1" style="font-weight: bold;" scope="col" id="20/11">20/11</th>
 <th class="gt_col_heading gt_columns_bottom_border gt_center" rowspan="1" colspan="1" style="font-weight: bold;" scope="col" id="20/12">20/12</th>
 <th class="gt_col_heading gt_columns_bottom_border gt_center" rowspan="1" colspan="1" style="font-weight: bold;" scope="col" id="21/1">21/1</th>
 <th class="gt_col_heading gt_columns_bottom_border gt_center" rowspan="1" colspan="1" style="font-weight: bold;" scope="col" id="21/2">21/2</th>
 <th class="gt_col_heading gt_columns_bottom_border gt_center" rowspan="1" colspan="1" style="font-weight: bold;" scope="col" id="21/3">21/3</th>
 <th class="gt_col_heading gt_columns_bottom_border gt_center" rowspan="1" colspan="1" style="font-weight: bold;" scope="col" id="21/4">21/4</th>
 <th class="gt_col_heading gt_columns_bottom_border gt_center" rowspan="1" colspan="1" style="font-weight: bold;" scope="col" id="21/5">21/5</th>
 <th class="gt_col_heading gt_columns_bottom_border gt_center" rowspan="1" colspan="1" style="font-weight: bold;" scope="col" id="21/6">21/6</th>
 <th class="gt_col_heading gt_columns_bottom_border gt_center" rowspan="1" colspan="1" style="font-weight: bold;" scope="col" id="21/7">21/7</th>
 <th class="gt_col_heading gt_columns_bottom_border gt_center" rowspan="1" colspan="1" style="font-weight: bold;" scope="col" id="21/8">21/8</th>
 <th class="gt_col_heading gt_columns_bottom_border gt_center" rowspan="1" colspan="1" style="font-weight: bold;" scope="col" id="21/9">21/9</th>
 <th class="gt_col_heading gt_columns_bottom_border gt_center" rowspan="1" colspan="1" style="font-weight: bold;" scope="col" id=" "> </th>
 <th class="gt_col_heading gt_columns_bottom_border gt_center" rowspan="1" colspan="1" style="font-weight: bold;" scope="col" id="Avg">Avg</th>
 </tr>
 </thead>
 <tbody class="gt_table_body">
 <tr><th id="stub_1_1" scope="row" class="gt_row gt_center gt_stub" style="font-weight: bold;">GAM.full</th>
<td headers="stub_1_1 20/10" class="gt_row gt_center" style="background-color: #8CC767; color: #000000;">.1239 (56.3)</td>
<td headers="stub_1_1 20/11" class="gt_row gt_center" style="background-color: #95CA66; color: #000000;">.0703 (58.5)</td>
<td headers="stub_1_1 20/12" class="gt_row gt_center" style="background-color: #86C467; color: #000000;">.0497 (56.6)</td>
<td headers="stub_1_1 21/1" class="gt_row gt_center" style="background-color: #7EC268; color: #000000;">.0532 (57.6)</td>
<td headers="stub_1_1 21/2" class="gt_row gt_center" style="background-color: #80C268; color: #000000;">.0988 (55.4)</td>
<td headers="stub_1_1 21/3" class="gt_row gt_center" style="background-color: #79C068; color: #000000;">.1202 (57.8)</td>
<td headers="stub_1_1 21/4" class="gt_row gt_center" style="background-color: #6EBC69; color: #000000;">.1447 (57.2)</td>
<td headers="stub_1_1 21/5" class="gt_row gt_center" style="background-color: #E5E45C; color: #000000;">.1746 (56.7)</td>
<td headers="stub_1_1 21/6" class="gt_row gt_center" style="background-color: #6CBC6A; color: #000000;">.1368 (56.3)</td>
<td headers="stub_1_1 21/7" class="gt_row gt_center" style="background-color: #94CA66; color: #000000;">.1537 (53.9)</td>
<td headers="stub_1_1 21/8" class="gt_row gt_center" style="background-color: #7AC068; color: #000000;">.1383 (60.2)</td>
<td headers="stub_1_1 21/9" class="gt_row gt_center" style="background-color: #68BB6A; color: #000000;">.1163 (56.9)</td>
<td headers="stub_1_1" class="gt_row gt_center" style="border-left-width: 1px; border-left-style: solid; border-left-color: transparent; border-right-width: 1px; border-right-style: solid; border-right-color: transparent; border-top-width: 1px; border-top-style: solid; border-top-color: transparent; border-bottom-width: 1px; border-bottom-style: solid; border-bottom-color: transparent;"></td>
<td headers="stub_1_1 Avg" class="gt_row gt_center" style="background-color: #84C467; color: #000000;">.1150 (56.9)</td></tr>
 <tr><th id="stub_1_2" scope="row" class="gt_row gt_center gt_stub" style="font-weight: bold;">GAM.red</th>
<td headers="stub_1_2 20/10" class="gt_row gt_center" style="background-color: #90C866; color: #000000;">.1241 (56.2)</td>
<td headers="stub_1_2 20/11" class="gt_row gt_center" style="background-color: #8CC667; color: #000000;">.0700 (58.7)</td>
<td headers="stub_1_2 20/12" class="gt_row gt_center" style="background-color: #81C368; color: #000000;">.0496 (56.6)</td>
<td headers="stub_1_2 21/1" class="gt_row gt_center" style="background-color: #90C866; color: #000000;">.0536 (57.3)</td>
<td headers="stub_1_2 21/2" class="gt_row gt_center" style="background-color: #8FC866; color: #000000;">.0994 (55.2)</td>
<td headers="stub_1_2 21/3" class="gt_row gt_center" style="background-color: #77C069; color: #000000;">.1201 (57.8)</td>
<td headers="stub_1_2 21/4" class="gt_row gt_center" style="background-color: #90C866; color: #000000;">.1467 (56.6)</td>
<td headers="stub_1_2 21/5" class="gt_row gt_center" style="background-color: #EAE65B; color: #000000;">.1750 (56.6)</td>
<td headers="stub_1_2 21/6" class="gt_row gt_center" style="background-color: #8DC767; color: #000000;">.1385 (55.8)</td>
<td headers="stub_1_2 21/7" class="gt_row gt_center" style="background-color: #B3D463; color: #000000;">.1554 (53.4)</td>
<td headers="stub_1_2 21/8" class="gt_row gt_center" style="background-color: #6BBC6A; color: #000000;">.1374 (60.5)</td>
<td headers="stub_1_2 21/9" class="gt_row gt_center" style="background-color: #66BA6A; color: #000000;">.1162 (56.9)</td>
<td headers="stub_1_2" class="gt_row gt_center" style="border-left-width: 1px; border-left-style: solid; border-left-color: transparent; border-right-width: 1px; border-right-style: solid; border-right-color: transparent; border-top-width: 1px; border-top-style: solid; border-top-color: transparent; border-bottom-width: 1px; border-bottom-style: solid; border-bottom-color: transparent;"></td>
<td headers="stub_1_2 Avg" class="gt_row gt_center" style="background-color: #8EC766; color: #000000;">.1155 (56.7)</td></tr>
 <tr><th id="stub_1_3" scope="row" class="gt_row gt_center gt_stub" style="font-weight: bold;">DNN</th>
<td headers="stub_1_3 20/10" class="gt_row gt_center" style="background-color: #7AC068; color: #000000;">.1230 (56.6)</td>
<td headers="stub_1_3 20/11" class="gt_row gt_center" style="background-color: #98CB65; color: #000000;">.0704 (58.4)</td>
<td headers="stub_1_3 20/12" class="gt_row gt_center" style="background-color: #B7D562; color: #000000;">.0507 (55.7)</td>
<td headers="stub_1_3 21/1" class="gt_row gt_center" style="background-color: #B8D562; color: #000000;">.0545 (56.6)</td>
<td headers="stub_1_3 21/2" class="gt_row gt_center" style="background-color: #E4E45C; color: #000000;">.1027 (53.7)</td>
<td headers="stub_1_3 21/3" class="gt_row gt_center" style="background-color: #D9E05E; color: #000000;">.1251 (56.1)</td>
<td headers="stub_1_3 21/4" class="gt_row gt_center" style="background-color: #DDE25D; color: #000000;">.1514 (55.2)</td>
<td headers="stub_1_3 21/5" class="gt_row gt_center" style="background-color: #66BA6A; color: #000000;">.1653 (59.0)</td>
<td headers="stub_1_3 21/6" class="gt_row gt_center" style="background-color: #FFE54E; color: #000000;">.1474 (52.9)</td>
<td headers="stub_1_3 21/7" class="gt_row gt_center" style="background-color: #96CA66; color: #000000;">.1538 (53.9)</td>
<td headers="stub_1_3 21/8" class="gt_row gt_center" style="background-color: #B6D562; color: #000000;">.1422 (59.1)</td>
<td headers="stub_1_3 21/9" class="gt_row gt_center" style="background-color: #EBE65B; color: #000000;">.1225 (54.6)</td>
<td headers="stub_1_3" class="gt_row gt_center" style="border-left-width: 1px; border-left-style: solid; border-left-color: transparent; border-right-width: 1px; border-right-style: solid; border-right-color: transparent; border-top-width: 1px; border-top-style: solid; border-top-color: transparent; border-bottom-width: 1px; border-bottom-style: solid; border-bottom-color: transparent;"></td>
<td headers="stub_1_3 Avg" class="gt_row gt_center" style="background-color: #B6D562; color: #000000;">.1174 (56.0)</td></tr>
 <tr><th id="stub_1_4" scope="row" class="gt_row gt_center gt_stub" style="font-weight: bold;">Combination</th>
<td headers="stub_1_4 20/10" class="gt_row gt_center" style="background-color: #66BA6A; color: #000000;">.1221 (56.9)</td>
<td headers="stub_1_4 20/11" class="gt_row gt_center" style="background-color: #66BA6A; color: #000000;">.0689 (59.3)</td>
<td headers="stub_1_4 20/12" class="gt_row gt_center" style="background-color: #66BA6A; color: #000000;">.0491 (57.1)</td>
<td headers="stub_1_4 21/1" class="gt_row gt_center" style="background-color: #66BA6A; color: #000000;">.0527 (58.0)</td>
<td headers="stub_1_4 21/2" class="gt_row gt_center" style="background-color: #66BA6A; color: #000000;">.0979 (55.8)</td>
<td headers="stub_1_4 21/3" class="gt_row gt_center" style="background-color: #66BA6A; color: #000000;">.1193 (58.1)</td>
<td headers="stub_1_4 21/4" class="gt_row gt_center" style="background-color: #66BA6A; color: #000000;">.1443 (57.3)</td>
<td headers="stub_1_4 21/5" class="gt_row gt_center" style="background-color: #92C966; color: #000000;">.1684 (58.2)</td>
<td headers="stub_1_4 21/6" class="gt_row gt_center" style="background-color: #66BA6A; color: #000000;">.1365 (56.4)</td>
<td headers="stub_1_4 21/7" class="gt_row gt_center" style="background-color: #66BA6A; color: #000000;">.1512 (54.6)</td>
<td headers="stub_1_4 21/8" class="gt_row gt_center" style="background-color: #66BA6A; color: #000000;">.1371 (60.6)</td>
<td headers="stub_1_4 21/9" class="gt_row gt_center" style="background-color: #6BBB6A; color: #000000;">.1164 (56.9)</td>
<td headers="stub_1_4" class="gt_row gt_center" style="border-left-width: 1px; border-left-style: solid; border-left-color: transparent; border-right-width: 1px; border-right-style: solid; border-right-color: transparent; border-top-width: 1px; border-top-style: solid; border-top-color: transparent; border-bottom-width: 1px; border-bottom-style: solid; border-bottom-color: transparent;"></td>
<td headers="stub_1_4 Avg" class="gt_row gt_center" style="background-color: #66BA6A; color: #000000;">.1137 (57.4)</td></tr>
 <tr><th id="stub_1_5" scope="row" class="gt_row gt_center gt_stub" style="font-weight: bold;">GAM.noWeather</th>
<td headers="stub_1_5 20/10" class="gt_row gt_center" style="background-color: #FFEC57; color: #000000;">.1301 (54.1)</td>
<td headers="stub_1_5 20/11" class="gt_row gt_center" style="background-color: #A2CE64; color: #000000;">.0707 (58.2)</td>
<td headers="stub_1_5 20/12" class="gt_row gt_center" style="background-color: #BCD762; color: #000000;">.0508 (55.6)</td>
<td headers="stub_1_5 21/1" class="gt_row gt_center" style="background-color: #83C368; color: #000000;">.0533 (57.5)</td>
<td headers="stub_1_5 21/2" class="gt_row gt_center" style="background-color: #FFE44D; color: #000000;">.1058 (52.3)</td>
<td headers="stub_1_5 21/3" class="gt_row gt_center" style="background-color: #DCE15D; color: #000000;">.1253 (56.0)</td>
<td headers="stub_1_5 21/4" class="gt_row gt_center" style="background-color: #97CA66; color: #000000;">.1471 (56.5)</td>
<td headers="stub_1_5 21/5" class="gt_row gt_center" style="background-color: #FFE048; color: #000000;">.1822 (54.8)</td>
<td headers="stub_1_5 21/6" class="gt_row gt_center" style="background-color: #AAD164; color: #000000;">.1401 (55.3)</td>
<td headers="stub_1_5 21/7" class="gt_row gt_center" style="background-color: #E6E55C; color: #000000;">.1584 (52.5)</td>
<td headers="stub_1_5 21/8" class="gt_row gt_center" style="background-color: #B9D662; color: #000000;">.1424 (59.1)</td>
<td headers="stub_1_5 21/9" class="gt_row gt_center" style="background-color: #FFE852; color: #000000;">.1249 (53.7)</td>
<td headers="stub_1_5" class="gt_row gt_center" style="border-left-width: 1px; border-left-style: solid; border-left-color: transparent; border-right-width: 1px; border-right-style: solid; border-right-color: transparent; border-top-width: 1px; border-top-style: solid; border-top-color: transparent; border-bottom-width: 1px; border-bottom-style: solid; border-bottom-color: transparent;"></td>
<td headers="stub_1_5 Avg" class="gt_row gt_center" style="background-color: #DDE25D; color: #000000;">.1193 (55.3)</td></tr>
 <tr><th id="stub_1_6" scope="row" class="gt_row gt_center gt_stub" style="font-weight: bold;">GAM.simple</th>
<td headers="stub_1_6 20/10" class="gt_row gt_center" style="background-color: #FFB326; color: #000000;">.1531 (46.0)</td>
<td headers="stub_1_6 20/11" class="gt_row gt_center" style="background-color: #FFA428; color: #000000;">.0940 (44.5)</td>
<td headers="stub_1_6 20/12" class="gt_row gt_center" style="background-color: #FFC127; color: #000000;">.0584 (49.0)</td>
<td headers="stub_1_6 21/1" class="gt_row gt_center" style="background-color: #FFB126; color: #000000;">.0672 (46.5)</td>
<td headers="stub_1_6 21/2" class="gt_row gt_center" style="background-color: #FFA826; color: #000000;">.1267 (42.9)</td>
<td headers="stub_1_6 21/3" class="gt_row gt_center" style="background-color: #FFB526; color: #000000;">.1500 (47.3)</td>
<td headers="stub_1_6 21/4" class="gt_row gt_center" style="background-color: #FFB626; color: #000000;">.1797 (46.8)</td>
<td headers="stub_1_6 21/5" class="gt_row gt_center" style="background-color: #FFB526; color: #000000;">.2093 (48.1)</td>
<td headers="stub_1_6 21/6" class="gt_row gt_center" style="background-color: #FFAF26; color: #000000;">.1733 (44.7)</td>
<td headers="stub_1_6 21/7" class="gt_row gt_center" style="background-color: #FFB226; color: #000000;">.1873 (43.8)</td>
<td headers="stub_1_6 21/8" class="gt_row gt_center" style="background-color: #FFC127; color: #000000;">.1671 (52.0)</td>
<td headers="stub_1_6 21/9" class="gt_row gt_center" style="background-color: #FFBB27; color: #000000;">.1413 (47.6)</td>
<td headers="stub_1_6" class="gt_row gt_center" style="border-left-width: 1px; border-left-style: solid; border-left-color: transparent; border-right-width: 1px; border-right-style: solid; border-right-color: transparent; border-top-width: 1px; border-top-style: solid; border-top-color: transparent; border-bottom-width: 1px; border-bottom-style: solid; border-bottom-color: transparent;"></td>
<td headers="stub_1_6 Avg" class="gt_row gt_center" style="background-color: #FFB526; color: #000000;">.1423 (46.7)</td></tr>
 <tr><th id="stub_1_7" scope="row" class="gt_row gt_center gt_stub" style="font-weight: bold;">Naive</th>
<td headers="stub_1_7 20/10" class="gt_row gt_center" style="background-color: #EE5250; color: #FFFFFF;">.2833</td>
<td headers="stub_1_7 20/11" class="gt_row gt_center" style="background-color: #EE5250; color: #FFFFFF;">.1693</td>
<td headers="stub_1_7 20/12" class="gt_row gt_center" style="background-color: #EE5250; color: #FFFFFF;">.1144</td>
<td headers="stub_1_7 21/1" class="gt_row gt_center" style="background-color: #EE5250; color: #FFFFFF;">.1255</td>
<td headers="stub_1_7 21/2" class="gt_row gt_center" style="background-color: #EE5250; color: #FFFFFF;">.2217</td>
<td headers="stub_1_7 21/3" class="gt_row gt_center" style="background-color: #EE5250; color: #FFFFFF;">.2847</td>
<td headers="stub_1_7 21/4" class="gt_row gt_center" style="background-color: #EE5250; color: #FFFFFF;">.3380</td>
<td headers="stub_1_7 21/5" class="gt_row gt_center" style="background-color: #EE5250; color: #FFFFFF;">.4029</td>
<td headers="stub_1_7 21/6" class="gt_row gt_center" style="background-color: #EE5250; color: #FFFFFF;">.3131</td>
<td headers="stub_1_7 21/7" class="gt_row gt_center" style="background-color: #EE5250; color: #FFFFFF;">.3334</td>
<td headers="stub_1_7 21/8" class="gt_row gt_center" style="background-color: #EE5250; color: #FFFFFF;">.3478</td>
<td headers="stub_1_7 21/9" class="gt_row gt_center" style="background-color: #EE5250; color: #FFFFFF;">.2699</td>
<td headers="stub_1_7" class="gt_row gt_center" style="border-left-width: 1px; border-left-style: solid; border-left-color: transparent; border-right-width: 1px; border-right-style: solid; border-right-color: transparent; border-top-width: 1px; border-top-style: solid; border-top-color: transparent; border-bottom-width: 1px; border-bottom-style: solid; border-bottom-color: transparent;"></td>
<td headers="stub_1_7 Avg" class="gt_row gt_center" style="background-color: #EE5250; color: #FFFFFF;">.2670</td></tr>
 </tbody>
 
 
</table>
</div>

---

# Wrap-Up

Estimating high-resolution elecricity peak demand using lower-resolution data:

- .bold[GAM.full] and .bold[GAM.red] perform similar
- .bold[DNN] beats .bold[GAM.full] in some Months
- .bold[Combination of GAM.full, GAM.red, and DNN] performs best

- Weather variables improve the skill score by 1.5 percentage points on average
- .bold[DNN] performs better at predicting maximum peak loads

]

We won the competition.

- 42.6% vs. 43.6% (second place)
- Using slightly different model

.center[
<center>
<img src="frame.png">
</center>
[berrisch.biz/slides/23_09_stat_woche](https://berrisch.biz/slides/23_09_stat_woche/)
]

]

<a href="https://github.com/BerriJ" class="github-corner" aria-label="View source on Github"><svg width="80" height="80" viewBox="0 0 250 250" style="fill:#f2f2f2; color:#212121; position: absolute; top: 0; border: 0; right: 0;" aria-hidden="true"><path d="M0,0 L115,115 L130,115 L142,142 L250,250 L250,0 Z"></path><path d="M128.3,109.0 C113.8,99.7 119.0,89.6 119.0,89.6 C122.0,82.7 120.5,78.6 120.5,78.6 C119.2,72.0 123.4,76.3 123.4,76.3 C127.3,80.9 125.5,87.3 125.5,87.3 C122.9,97.6 130.6,101.9 134.4,103.2" fill="currentColor" style="transform-origin: 130px 106px;" class="octo-arm"></path><path d="M115.0,115.0 C114.9,115.1 118.7,116.5 119.8,115.4 L133.7,101.6 C136.9,99.2 139.9,98.4 142.2,98.6 C133.8,88.0 127.5,74.4 143.8,58.0 C148.5,53.4 154.0,51.2 159.7,51.0 C160.3,49.4 163.2,43.6 171.4,40.1 C171.4,40.1 176.1,42.5 178.8,56.2 C183.1,58.6 187.2,61.8 190.9,65.4 C194.5,69.0 197.7,73.2 200.1,77.6 C213.8,80.2 216.3,84.9 216.3,84.9 C212.7,93.1 206.9,96.0 205.4,96.6 C205.1,102.4 203.0,107.8 198.3,112.5 C181.9,128.9 168.3,122.5 157.7,114.1 C157.9,116.9 156.7,120.9 152.7,124.9 L141.0,136.5 C139.8,137.7 141.6,141.9 141.8,141.8 Z" fill="currentColor" class="octo-body"></path></svg></a><style>.github-corner:hover .octo-arm{animation:octocat-wave 560ms ease-in-out}@keyframes octocat-wave{0%,100%{transform:rotate(0)}20%,60%{transform:rotate(-25deg)}40%,80%{transform:rotate(10deg)}}@media (max-width:500px){.github-corner:hover .octo-arm{animation:none}.github-corner .octo-arm{animation:octocat-wave 560ms ease-in-out}}</style>

---
name:references

# References 1

<cite><a id='bib-akiba2019optuna'></a><a href="#cite-akiba2019optuna">Akiba, T., S. Sano, T. Yanase, et al.</a>
(2019).
&ldquo;Optuna: A next-generation hyperparameter optimization framework&rdquo;.
In: 
Proceedings of the 25th ACM SIGKDD international conference on knowledge discovery &amp; data mining.
, pp. 2623&ndash;2631.</cite>

<cite><a id='bib-berrisch2023high'></a><a href="#cite-berrisch2023high">Berrisch, J., M. Narajewski, and F. Ziel</a>
(2023).
&ldquo;High-resolution peak demand estimation using generalized additive models and deep neural networks&rdquo;.
In: Energy and AI 13, p. 100236.</cite>

<cite><a id='bib-szekely2007measuring'></a><a href="#cite-szekely2007measuring">Székely, G. J., M. L. Rizzo, and N. K. Bakirov</a>
(2007).
&ldquo;Measuring and testing dependence by correlation of distances&rdquo;.
In: The annals of statistics 35.6, pp. 2769&ndash;2794.</cite>