--- title: Kaggle-Competition-Favorita keywords: fastai sidebar: home_sidebar nb_path: "nbs/data_datasets__favorita.ipynb" ---
%%html
<style> table {float:left} </style>
The evaluation metric of the Favorita Kaggle competition was the normalized weighted root mean squared logarithmic error (NWRMSLE). Perishable items have a score weight of 1.25; otherwise, the weight is 1.0.
{% raw %} $$ NWRMSLE = \sqrt{\frac{\sum^{n}_{i=1} w_{i}\left(log(\hat{y}_{i}+1) - log(y_{i}+1)\right)^{2}}{\sum^{n}_{i=1} w_{i}}}$$ {% endraw %}
Kaggle Competition Forecasting Methods | 16D ahead NWRMSLE | |
---|---|---|
LGBM [1] | 0.5091 | |
Seq2Seq WaveNet [2] | 0.5129 |
Given that the test set is unavailable, some papers have chosen a different data partition to work on the dataset. The following table uses the favorita dataset with 90 days for training and predicts 30 days into the future for the logarithm of sales. The evaluation metric used in other papers was the quantile risk for P50 (normalized quantile loss):
{% raw %} $$ QL-risk = \frac{2 \sum_{y_{t} \in \Omega } \sum^{\tau_{max}}_{\tau=t} q (y_{t+\tau}-\hat{y}^{(q)}_{t+\tau})_{+} + (1-q) (\hat{y}^{(q)}_{t+\tau}-y_{t+\tau})_{+} }{\sum_{y_{t} \in \Omega } \sum^{\tau_{max}}_{\tau=1} | y_{t} | }$$ {% endraw %}
Forecasting Methods | 30D ahead P50 QL-risk | |
---|---|---|
MQTransformer [4] | 0.323 | |
TFT [3] | 0.354 |