Multiperiod Forecasts

In-sample Forecasts

An in-sample forecast uses the fitted model to derive the predicted values within the period used to estimate model parameters. In-sample forecast errors are residuals generated from a fitted-time series model. For instance, if we use a linear trend to estimate the inflation rate using data from 2004 to 2018, the in-sample errors are the residuals from 2004 to 2018.

In sample forecast errors are expressed as: \(\text{y}_{\text{t}}-\hat{\text{y}}_{\text{t}}\) where \(t\) is an observation within the sample period.

Out-of-sample Forecast Errors

An out-of-sample forecast uses the estimated model parameters to forecast values outside of the sample period. An out-of-sample forecast error is calculated by taking the difference between the predicted values from outside the sample period and the realized/actual values.

For example, suppose we use the same model to predict the inflation rate outside this model’s period (2004 to 2018). In this case, the difference between the forecasted and actual inflation rate is an out-of-sample forecast error. Models are selected based on out-of-sample forecasting error. This is because the future is always out-of-sample.

A model’s accuracy in forecasting out-of-sample values is assessed using the root mean squared error (RMSE). RMSE is the square root of the mean squared error. The model with the smallest RMSE is seen as the most accurate, as it is perceived to have better predictive power in the future.

The following steps are used to assess model accuracy using RMSE:

• Calculate the residuals: This is done by taking the difference between the actual and the forecasted value;
• Square the residuals;
• Sum the squared residuals;
• Determine the average of the squared residuals; and
• Take the square root of the average of the squared residuals.

Example: Root Mean Squared Error (RMSE)

An analyst uses two models to analyze the same time-series data. The mean squared error for the first model, AR(1), is 6.9591, and that of the second model AR(2), is 5.6217. We will use the RMSE criterion to determine the most accurate model in forecasting out-of-sample values.

Solution

We first calculate the RMSE of both models and choose the model with the smallest RMSE

\(\text{AR}(1):\text{RMSE}=\sqrt{6.9591}=3.1558\)

\(\text{AR}(2): \text{RMSE}=\sqrt{5.6217}=2.3710\)

AR(2) has the smallest RMSE. Thus, it is the most accurate in forecasting out-of-sample values.

Question

The following table gives the actual and the forecasted change in the log of the price of a certain commodity for four years. The observations after 2016 are out of sample

$$\small{\begin{array}{c|c|c}\textbf{Year} & {\textbf{Actual values of}\\ \textbf{changes in prices}} & {\textbf{Forecast values of}\\ \textbf{changes in prices}}\\ \hline 2017 & 0.1275 & 0.1323 \\ \hline 2018 & 0.0336 & 0.1267 \\ \hline2019 & -0.3468 & 0.1239 \\ \hline 2020 & -0.093 & 0.1228\\ \end{array}}$$

The RMSE for the out-of-sample forecast errors is closest to:

1.  6.92%.
2. 7.84%.
3. 26.31%.

Solution

The correct answer is C.

The RMSE of the out-of-sample forecast errors is 26.31% percent. The out-of-sample error refers to the difference between the realized value and the forecasted value of change in the price for years beyond the estimation period. In this case, the out-of-sample period is 2017 to 2020.

The calculations of RMSE are shown in the following table:

$$\small{\begin{array}{c|c|c|c|c} \textbf{Year} & {\textbf{Actual values of}\\ \textbf{changes in prices}} & {\textbf{Forecast values of}\\ \textbf{changes in prices}} & \textbf{Residuals} & \textbf{Residuals}^{2}\\ \hline2017 & 0.1275 & 0.1323 & -0.0048 & 0.00002 \\ \hline2018 & 0.0336 & 0.1267 & -0.0931 & 0.00867 \\ \hline2019 & -0.3468 & 0.1239 & -0.4707 & 0.22156 \\ \hline2020 & -0.093 & 0.1228 & -0.2158 & 0.04657 \\ \hline & & & \textbf{Mean} & \textbf{0.06920}\\ \hline& & & \textbf{RMSE} & \textbf{26.31%}\\  \end{array}}$$

Reading 5: Time Series Analysis

LOS 5 (g) Contrast in-sample and out-of-sample forecasts and compare the forecasting accuracy of different time-series models based on the root mean squared error criterion.