forecastML Overview

Nickalus Redell



The purpose of forecastML is to provide a series of functions and visualizations that simplify the process of multi-step-ahead direct forecasting with standard machine learning algorithms. It’s a wrapper package aimed at providing maximum flexibility in model-building–choose any machine learning algorithm from any R package–while helping the user quickly assess the (a) accuracy, (b) stability, and (c) generalizability of grouped (i.e., multiple related time-series) and ungrouped single-outcome forecasts produced from potentially high-dimensional modeling datasets.

This package is inspired by Bergmeir, Hyndman, and Koo’s 2018 paper A note on the validity of cross-validation for evaluating autoregressive time series prediction. In particular, forecastML makes use of

to build and evaluate high-dimensional forecast models without having to use methods that are time-series specific.

The following quote from Bergmeir et al.’s article nicely sums up the aim of this package:

“When purely (non-linear, nonparametric) autoregressive methods are applied to forecasting problems, as is often the case (e.g., when using Machine Learning methods), the aforementioned problems of CV are largely irrelevant, and CV can and should be used without modification, as in the independent case.”

Direct forecasting

In contrast to the recursive or iterated method for producing multi-step-ahead forecasts used in traditional forecasting methods like ARIMA, direct forecasting involves creating a series of distinct horizon-specific models. Though several hybrid methods exist for producing multi-step forecasts, the simple direct forecasting method with lagged features used in forecastML let’s us avoid the exponentially more difficult problem of having to “predict the predictors” for forecast horizons beyond 1-step-ahead.

Below are some resources for learning more about multi-step forecasting strategies:

The animation below shows how historical data is used to create a 1-to-12-step-ahead forecast for a 12-step-horizon forecast model using lagged predictors or features. Though feature lags greater than 12 steps can be used to make use of additional historical predictive information, a 12-step-horizon direct forecast model requires feature lags >= 12. This animation is roughly equivalent to how a 12-period seasonal ARIMA(0, 0, 0)(1, 0, 0) model uses historical data to produce forecasts.

Package Features

  1. Transform datasets for modeling by creating various patterns of lagged features for user-specified forecast horizons with forecastML::create_lagged_df

  2. Create datasets for evaluating forecast models using nested cross-validation with forecastML::create_windows

  3. Train and evaluate machine learning models for forecasting with forecastML::train_model.

  4. Assess forecast accuracy at different forecast horizons with forecastML::return_error.

  5. Assess hyperparameter stability with forecastML::return_hyper.

  6. Create datasets of lagged features for direct forecasting.


In this walkthrough of forecastML we’ll compare the forecast performance of two machine learning methods, LASSO and Random Forest, across forecast horizons using the Seatbelts dataset from the datasets package.

Here’s a summary of the problem at hand:

Install forecastML

Load packages and data

Train-test split

Data Preparation


We’ll create a list of datasets, one for each forecast horizon, with lagged values for each feature. The lookback argument in forecastML::create_lagged_df() specifies the feature lags in dataset rows.

Let’s view the modeling dataset for a forecast horizon of 6. Notice that “lag” has been appended to all model features. Grouped and static features, covered in the “grouped_forecast” vignette, keep their original names.