The purpose of `forecastML`

is to provide a series of functions and visualizations that simplify the process of **multi-step-ahead direct forecasting with standard machine learning algorithms**. It’s a wrapper package aimed at providing maximum flexibility in model-building–**choose any machine learning algorithm from any R package**–while helping the user quickly assess the (a) accuracy, (b) stability, and (c) generalizability of grouped (i.e., multiple related time-series) and ungrouped single-outcome forecasts produced from potentially high-dimensional modeling datasets.

This package is inspired by Bergmeir, Hyndman, and Koo’s 2018 paper A note on the validity of cross-validation for evaluating autoregressive time series prediction. In particular, `forecastML`

makes use of

**lagged features**,**simple wrapper functions that support models from any**,`R`

package**nested cross-validation**with (a) user-specified standard cross-validation in the inner loop and (b) block-contiguous validation datasets in the outer loop, and**parallel processing**with the`future`

package

to build and evaluate high-dimensional forecast models **without having to use methods that are time-series specific**.

The following quote from Bergmeir et al.’s article nicely sums up the aim of this package:

“When purely (non-linear, nonparametric) autoregressive methods are applied to forecasting problems, as is often the case (e.g., when using Machine Learning methods), the aforementioned problems of CV are largely irrelevant, and CV can and should be used without modification, as in the independent case.”

In contrast to the recursive or iterated method for producing multi-step-ahead forecasts used in traditional forecasting methods like ARIMA, direct forecasting involves creating a series of distinct horizon-specific models. Though several hybrid methods exist for producing multi-step forecasts, the simple direct forecasting method with lagged features used in `forecastML`

let’s us avoid the exponentially more difficult problem of having to “predict the predictors” for forecast horizons beyond 1-step-ahead.

Below are some resources for learning more about multi-step forecasting strategies:

- A review and comparison of strategies for multi-step-ahead time series forecasting based on the NN5 forecasting competition
- A comparison of direct and iterated multistep AR methods for forecasting macroeconomic time series

The **animation below** shows how historical data is used to create a 1-to-12-step-ahead forecast for a 12-step-horizon forecast model using lagged predictors or features. Though feature lags greater than 12 steps can be used to make use of additional historical predictive information, a 12-step-horizon direct forecast model requires feature lags >= 12. This animation is roughly equivalent to how a 12-period seasonal ARIMA(0, 0, 0)(1, 0, 0) model uses historical data to produce forecasts.

**Transform datasets**for modeling by creating various patterns of lagged features for user-specified forecast horizons with`forecastML::create_lagged_df`

Create datasets for evaluating forecast models using

**nested cross-validation**with`forecastML::create_windows`

Train and

**evaluate machine learning models for forecasting**with`forecastML::train_model`

.Assess

**forecast accuracy**at different forecast horizons with`forecastML::return_error`

.Assess

**hyperparameter stability**with`forecastML::return_hyper`

.Create datasets of lagged features for

**direct forecasting**.

In this walkthrough of `forecastML`

we’ll compare the forecast performance of two machine learning methods, LASSO and Random Forest, across forecast horizons using the Seatbelts dataset from the `datasets`

package.

Here’s a summary of the problem at hand:

**Outcome:**`DriversKilled`

- car drivers killed per month in the UK.

**Predcitors:**`DriversKilled`

- car drivers killed per month in the UK.`kms`

- a measure of distance driven.`PetrolPrice`

- the price of gas.`law`

- A binary indicator of the presence of a seatbelt law.

**Forecast:**- Model training - The first 15 years of the monthly dataset.
- Model testing - The last year of the monthly dataset.

- We’ll build our models on
`data_train`

and evaluate their out-of-sample performance on`data_test`

.

```
ts_frequency <- 12 # monthly time-series
data_train <- data[1:(nrow(data) - ts_frequency), ]
data_test <- data[(nrow(data) - ts_frequency + 1):nrow(data), ]
p <- ggplot(data, aes(x = 1:nrow(data), y = DriversKilled))
p <- p + geom_line()
p <- p + geom_vline(xintercept = nrow(data_train), color = "red", size = 1.1)
p <- p + theme_bw() + xlab("Index")
p
```

We’ll create a list of datasets, one for each forecast horizon, with lagged values for each feature. The `lookback`

argument in `forecastML::create_lagged_df()`

specifies the feature lags in dataset rows.

```
horizons <- c(1, 3, 6, 9, 12)
lookback <- 1:15
data_list <- forecastML::create_lagged_df(data_train, type = "train",
outcome_col = 1, lookback = lookback,
horizons = horizons)
```

Let’s view the modeling dataset for a forecast horizon of 6. Notice that “*lag*