Multiperiod Forecasts

Multiperiod Forecasts

In-sample Forecasts

An in-sample forecast uses the fitted model to derive the predicted values within the period used to estimate model parameters. In-sample forecast errors are residuals generated from a fitted-time series model. For instance, if we use a linear trend to estimate the inflation rate using data from 2004 to 2018, the in-sample errors are the residuals from 2004 to 2018.

In sample forecast errors are expressed as: \(\text{y}_{\text{t}}-\hat{\text{y}}_{\text{t}}\) where \(t\) is an observation within the sample period.

Out-of-sample Forecast Errors

An out-of-sample forecast uses the estimated model parameters to forecast values outside of the sample period. An out-of-sample forecast error is calculated by taking the difference between the predicted values from outside the sample period and the realized/actual values.

For example, suppose we use the same model to predict the inflation rate outside this model’s period (2004 to 2018). In this case, the difference between the forecasted and actual inflation rate is an out-of-sample forecast error. Models are selected based on out-of-sample forecasting error. This is because the future is always out-of-sample.

A model’s accuracy in forecasting out-of-sample values is assessed using the root mean squared error (RMSE). RMSE is the square root of the mean squared error. The model with the smallest RMSE is seen as the most accurate, as it is perceived to have better predictive power in the future.

The following steps are used to assess model accuracy using RMSE:

  • Calculate the residuals: This is done by taking the difference between the actual and the forecasted value;
  • Square the residuals;
  • Sum the squared residuals;
  • Determine the average of the squared residuals; and
  • Take the square root of the average of the squared residuals.

Example: Root Mean Squared Error (RMSE)

An analyst uses two models to analyze the same time-series data. The mean squared error for the first model, AR(1), is 6.9591, and that of the second model AR(2), is 5.6217. We will use the RMSE criterion to determine the most accurate model in forecasting out-of-sample values.

Solution

We first calculate the RMSE of both models and choose the model with the smallest RMSE

\(\text{AR}(1):\text{RMSE}=\sqrt{6.9591}=3.1558\)

\(\text{AR}(2): \text{RMSE}=\sqrt{5.6217}=2.3710\)

AR(2) has the smallest RMSE. Thus, it is the most accurate in forecasting out-of-sample values.

Question

The following table gives the actual and the forecasted change in the log of the price of a certain commodity for four years. The observations after 2016 are out of sample

$$\small{\begin{array}{c|c|c}\textbf{Year} & {\textbf{Actual values of}\\ \textbf{changes in prices}} & {\textbf{Forecast values of}\\ \textbf{changes in prices}}\\ \hline 2017 & 0.1275 & 0.1323 \\ \hline 2018 & 0.0336 & 0.1267 \\ \hline2019 & -0.3468 & 0.1239 \\ \hline 2020 & -0.093 & 0.1228\\ \end{array}}$$

The RMSE for the out-of-sample forecast errors is closest to:

  1.  6.92%.
  2. 7.84%.
  3. 26.31%.

Solution

The correct answer is C.

The RMSE of the out-of-sample forecast errors is 26.31% percent. The out-of-sample error refers to the difference between the realized value and the forecasted value of change in the price for years beyond the estimation period. In this case, the out-of-sample period is 2017 to 2020.

The calculations of RMSE are shown in the following table:

$$\small{\begin{array}{c|c|c|c|c} \textbf{Year} & {\textbf{Actual values of}\\ \textbf{changes in prices}} & {\textbf{Forecast values of}\\ \textbf{changes in prices}} & \textbf{Residuals} & \textbf{Residuals}^{2}\\ \hline2017 & 0.1275 & 0.1323 & -0.0048 & 0.00002 \\ \hline2018 & 0.0336 & 0.1267 & -0.0931 & 0.00867 \\ \hline2019 & -0.3468 & 0.1239 & -0.4707 & 0.22156 \\ \hline2020 & -0.093 & 0.1228 & -0.2158 & 0.04657 \\ \hline & & & \textbf{Mean} & \textbf{0.06920}\\ \hline& & & \textbf{RMSE} & \textbf{26.31%}\\  \end{array}}$$

Reading 5: Time Series Analysis

LOS 5 (g) Contrast in-sample and out-of-sample forecasts and compare the forecasting accuracy of different time-series models based on the root mean squared error criterion.

 

Shop CFA® Exam Prep

Offered by AnalystPrep

Featured Shop FRM® Exam Prep Learn with Us

    Subscribe to our newsletter and keep up with the latest and greatest tips for success
    Shop Actuarial Exams Prep Shop GMAT® Exam Prep


    Daniel Glyn
    Daniel Glyn
    2021-03-24
    I have finished my FRM1 thanks to AnalystPrep. And now using AnalystPrep for my FRM2 preparation. Professor Forjan is brilliant. He gives such good explanations and analogies. And more than anything makes learning fun. A big thank you to Analystprep and Professor Forjan. 5 stars all the way!
    michael walshe
    michael walshe
    2021-03-18
    Professor James' videos are excellent for understanding the underlying theories behind financial engineering / financial analysis. The AnalystPrep videos were better than any of the others that I searched through on YouTube for providing a clear explanation of some concepts, such as Portfolio theory, CAPM, and Arbitrage Pricing theory. Watching these cleared up many of the unclarities I had in my head. Highly recommended.
    Nyka Smith
    Nyka Smith
    2021-02-18
    Every concept is very well explained by Nilay Arun. kudos to you man!
    Badr Moubile
    Badr Moubile
    2021-02-13
    Very helpfull!
    Agustin Olcese
    Agustin Olcese
    2021-01-27
    Excellent explantions, very clear!
    Jaak Jay
    Jaak Jay
    2021-01-14
    Awesome content, kudos to Prof.James Frojan
    sindhushree reddy
    sindhushree reddy
    2021-01-07
    Crisp and short ppt of Frm chapters and great explanation with examples.