--- title: M4 dataset keywords: fastai sidebar: home_sidebar summary: "Download and evaluate the M4 dataset." description: "Download and evaluate the M4 dataset." nb_path: "nbs/data_datasets__m4.ipynb" ---
{% raw %}
{% endraw %} {% raw %}
{% endraw %}

M4 meta information

{% raw %}

class Yearly[source]

Yearly(seasonality:int=1, horizon:int=6, freq:str='Y', name:str='Yearly', n_ts:int=23000)

Yearly(seasonality: int = 1, horizon: int = 6, freq: str = 'Y', name: str = 'Yearly', n_ts: int = 23000)

{% endraw %} {% raw %}

class Quarterly[source]

Quarterly(seasonality:int=4, horizon:int=8, freq:str='Q', name:str='Quarterly', n_ts:int=24000)

Quarterly(seasonality: int = 4, horizon: int = 8, freq: str = 'Q', name: str = 'Quarterly', n_ts: int = 24000)

{% endraw %} {% raw %}

class Monthly[source]

Monthly(seasonality:int=12, horizon:int=18, freq:str='M', name:str='Monthly', n_ts:int=48000)

Monthly(seasonality: int = 12, horizon: int = 18, freq: str = 'M', name: str = 'Monthly', n_ts: int = 48000)

{% endraw %} {% raw %}

class Weekly[source]

Weekly(seasonality:int=1, horizon:int=13, freq:str='W', name:str='Weekly', n_ts:int=359)

Weekly(seasonality: int = 1, horizon: int = 13, freq: str = 'W', name: str = 'Weekly', n_ts: int = 359)

{% endraw %} {% raw %}

class Daily[source]

Daily(seasonality:int=1, horizon:int=14, freq:str='D', name:str='Daily', n_ts:int=4227)

Daily(seasonality: int = 1, horizon: int = 14, freq: str = 'D', name: str = 'Daily', n_ts: int = 4227)

{% endraw %} {% raw %}

class Hourly[source]

Hourly(seasonality:int=24, horizon:int=48, freq:str='H', name:str='Hourly', n_ts:int=414)

Hourly(seasonality: int = 24, horizon: int = 48, freq: str = 'H', name: str = 'Hourly', n_ts: int = 414)

{% endraw %} {% raw %}

class Other[source]

Other(seasonality:int=1, horizon:int=8, freq:str='D', name:str='Other', n_ts:int=5000, included_groups:Tuple=('Weekly', 'Daily', 'Hourly'))

Other(seasonality: int = 1, horizon: int = 8, freq: str = 'D', name: str = 'Other', n_ts: int = 5000, included_groups: Tuple = ('Weekly', 'Daily', 'Hourly'))

{% endraw %} {% raw %}
{% endraw %} {% raw %}
{% endraw %}

Download data class

{% raw %}

class M4[source]

M4(source_url:str='https://raw.githubusercontent.com/Mcompetitions/M4-methods/master/Dataset/', naive2_forecast_url:str='https://github.com/Nixtla/m4-forecasts/raw/master/forecasts/submission-Naive2.zip')

M4(source_url: str = 'https://raw.githubusercontent.com/Mcompetitions/M4-methods/master/Dataset/', naive2_forecast_url: str = 'https://github.com/Nixtla/m4-forecasts/raw/master/forecasts/submission-Naive2.zip')

{% endraw %} {% raw %}
{% endraw %} {% raw %}
for group, meta in M4Info:
    if group != 'Hourly':
        continue
    df, *_ = M4.load(directory='data', group=group)
    n_series = len(np.unique(df.unique_id.values))

    display_str  = f'Group: {group} '
    display_str += f'n_series: {n_series}'
    print(display_str)
Group: Hourly n_series: 414
{% endraw %}

Evaluation class

{% raw %}

class M4Evaluation[source]

M4Evaluation()

{% endraw %} {% raw %}
{% endraw %}

URL-based evaluation

The method evaluate from the class M4Evaluation can receive a url of a benchmark uploaded to the M4 competiton.

The results compared to the on-the-fly evaluation were obtained from the official evaluation.

{% raw %}
esrnn_url = 'https://github.com/Nixtla/m4-forecasts/raw/master/forecasts/submission-118.zip'
esrnn_evaluation = M4Evaluation.evaluate('data', 'Hourly', esrnn_url)
# Test of the same evaluation as the original one
test_close(esrnn_evaluation['SMAPE'].item(), 9.328, eps=1e-3)
test_close(esrnn_evaluation['MASE'].item(), 0.893, eps=1e-3)
test_close(esrnn_evaluation['OWA'].item(), 0.440, eps=1e-3)
esrnn_evaluation
SMAPE MASE OWA
Hourly 9.328443 0.893046 0.440163
{% endraw %}

Numpy-based evaluation

Also the method evaluate can recevie a numpy array of forecasts.

{% raw %}
fforma_url = 'https://github.com/Nixtla/m4-forecasts/raw/master/forecasts/submission-245.zip'
fforma_forecasts = M4Evaluation.load_benchmark('data', 'Hourly', fforma_url)
fforma_evaluation = M4Evaluation.evaluate('data', 'Hourly', fforma_forecasts)
# Test of the same evaluation as the original one
test_close(fforma_evaluation['SMAPE'].item(), 11.506, eps=1e-3)
test_close(fforma_evaluation['MASE'].item(), 0.819, eps=1e-3)
test_close(fforma_evaluation['OWA'].item(), 0.484, eps=1e-3)
fforma_evaluation
SMAPE MASE OWA
Hourly 11.505702 0.818598 0.483841
{% endraw %}