Analyzing the Regression Output

Most of the regression analysis is done using statistical software. You are mainly supposed to interpret the regression output.

ANOVA Table

One of the outputs of multiple regression is the ANOVA table. The following shows the general structure of an Anova table.

$$\small{\begin{array}{c|c|c|c} \textbf{ANOVA} & \textbf{Df (Degrees of Freedom)} & \textbf{SS (Sum of Squares)} & \textbf{MSS (Mean Sum of Squares)}\\ \hline\text{Regression} & \text{k} & \text{RSS (Explained Variation)} & \text{MSR}\\ \hline\text{Residual} & n-(k+1) & \text{SSE (Unexplained Variation)} & \text{MSE}\\ \hline\text{Total} & n-1 & \text{SST (Total Variation)} &{}\\  \end{array}}$$

$$R^{2}= \frac{RSS}{SST}$$

$$F=\frac{MSR}{MSE}$$

$$SEE=\sqrt{MSE}$$

Where:

$$MSR=\frac{RSS}{k}$$

$$MSE=\frac{SSE}{n-k-1}$$

Exam tip: You might be required to fill in missing data and interpret the regression results at a given level of significance using the Anova Table.

Example: Interpreting Regression Output

Consider the following regression results generated from the multiple regression analysis of the price of the US Dollar index on the inflation rate and real rate of interest.

$$ \textbf{ANOVA Table} $$

$$\small{\begin{array}{lcccccc}\hline{}& \textbf{df} & \textbf{SS} & \textbf{MS} & \textbf{F} & \textbf{Significance F}\\ \hline\text{Regression} & 2 & 432.2520 & ? & ? & 0.0179\\ \text{Residual} & 7 & 200.6349 & ? & & {}\\ \text{Total} & 9 & 632.8869 & & &{}\\ \hline\end{array}}$$

$$\small{\begin{array}{lccccc}\hline{}& \textbf{Coefficients} & \textbf{Standard Error} & \textbf{t-stat} & \textbf{p-value}\\ \hline\text{Intercept} & 81 & 7.9659 & 10.1296 & 0.0000\\ \text{Inflation Rates} & -276 & 233.0748 & ? & 0.2753\\ \text{Real interest Rates} & 902 & 279.6949 & ? & 0.0145\\ \hline\end{array}}$$

Given the above information, the regression equation can be expressed as:

$$P=81-276INF+902IR$$

Where:

• \(P\) = Price of USDX
• \(INF\) = Inflation rate
• \(IR\) = Real interest rate

We can fill the missing data in the above table as follows:

Mean Regression Sum of Squares (MSR)

$$\begin{align*}MSR&=\frac{RSS}{k}\\&=\frac{432.2520}{2}\\&=216.126\end{align*}$$

Mean Squared Error (MSE)

$$\begin{align*}\text{MSE}&= \frac{SSE}{n-k-1}\\&=\frac{200.6349}{10-2-1}\\&=28.662\end{align*}$$

F-Statistic

$$\begin{align*}F&=\frac{MSR}{MSE}\\&=\frac{216.126}{28.662}\\&=7.5405\end{align*}$$

\(R^{2}\) and adjusted \(R^{2}\) can also be calculated as follows:

$$\begin{align*}R^{2}&=\frac{RSS}{SST}\\&=\frac{432.2520}{632.8869}\\&=68.30\%\end{align*}$$

$$\begin{align*}\text{Adjusted } R^{2}&=1-\bigg(\frac{n-1}{n-k-1})\bigg(1-R^{2})\\&=1-\frac{10-1}{10-2-1}(1-0.6830)\end{align*}$$

$$\text{Adjusted } R^{2}=0.5924=59.24\%$$

Test statistics

$$t=\frac{\widehat{b_{j}}-b_{Ho}}{S_{\widehat{S_{j}}}}$$

$$t_{INF}=\frac{-276-0}{233.0748}=-1.184$$

$$t_{IR}=\frac{902-0}{279.6949}=3.225$$

Significance of Individual Coefficients at the 5% Significance Level

T-tests and p-values can be used to test for the significance of slope coefficients.

p-value

The P-values can be observed directly from the regression output. Notice that only the p-value of the real interest rate is less than 0.05. This implies that the real rate of interest significantly influences the price of the US Dollar index.

t-test

A t-test involves testing the null hypothesis that the regression coefficient is zero against the alternative hypothesis that it is not.

\(H_{0}:b_j=0\) v.s. \(H_{a}:b_{j}≠0\)

Under the decision rule, the null hypothesis is rejected when the absolute value of the test statistic exceeds the critical value of t. The critical two-tailed t value at the 5% significance level is:

$$\begin{align*}t_{\frac{α}{2}, n-k-1}&=t_{\frac{0.05}{2},10-2-1}\\&=t_{0.025,7}\\&=2.3646\end{align*}$$

\(|t_{INF}|=1.184\) is less than the critical t-value of 2.3646. Thus, we cannot reject the null for inflation rates, and we conclude that the inflation rates do not make a statistically significant contribution to the variation in the prices of USDX at the 5% significance level.

On the other hand, \(|t_{IR}|=3.225\) is greater than the critical t-value of 2.3646. The null hypothesis for the real interest rate is therefore rejected. The conclusion is that the real interest rates make a significant contribution to the variation in USDX prices at the 5% significance level.

P-values and t-tests generate the same conclusions about the significance of the regression coefficients.

F-test

The F-test checks the regression’s generalized significance. The null hypothesis for a one-sided F-test is \(H_{0}:b_{INF}=b_{IR}=0\) against the alternative hypothesis: \(H_{1}: b_{INF}≠0\) or \(b_{IR}≠0\).

The critical value of F at the 5% significance level with \((10 – 2 – 1) = 7\) degrees of freedom \(F_{0.05, 2,7}\) is approximately 4.737. Since this is a none tailed test, we have used the 5% F-table. The decision rule is to reject the null hypothesis when \(F_{statistic}>F_{critical}\).

In this case, the \(F_{statistic} (7.5405)>F_{critical}(4.737)\). We, therefore, reject the null hypothesis and conclude that at least one of the independent variables significantly contributes to the dependent variable. Additionally, notice that the ANOVA table shows that F is significant since P-value \(<0.05\).

Question

Adil Suleman, CFA, wishes to establish the possible drivers of a company’s percentage return on capital (ROC). Suleman identifies performance measures such as the profit margin (%), sales, and debt ratio as possible drivers of ROC.

He then regresses the ROC against the three independent variables: Sales, profit margin, and debt ratio. The output from a statistical software package is presented in the following table.

$$ \textbf{ANOVA Table} $$

$$\small{\begin{array}{l|c|c} {}& \textbf{df} & \textbf{SS}\\ \hline\text{Regression} & 3 & 50.1055\\ \hline\text{Residual} & 21 & 30.0545\\ \hline\text{Total} & 24 & 80.1600\\  \end{array}}$$

With respect to the F-statistic, the most appropriate decision regarding testing the null hypothesis that all the independent variables are jointly equal to zero is to:

      A. Reject the null hypothesis because the F-statistic is greater than the critical F-value of 3.072.

      B. Fail to reject the null hypothesis because the F-statistic is smaller than the critical F-value of 3.072.

       C. Reject the null hypothesis since the F-statistic is greater than the critical F-value of 3.028.

Solution

The correct answer is A.

$$\begin{align*}F-\text{statistic}&=\frac{MSR}{MSE}=\frac{\frac{RSS}{k}}{\frac{SSE}{n-k-1}}\\&=\frac{\bigg(\frac{50.1055}{3}\bigg)}{\bigg(\frac{30.0545}{21}\bigg)}=\frac{16.7018}{1.43}\\&=11.67\end{align*}$$

With 3 and 21 degrees of freedom, the critical value for a 5% F-test is 3.072.

As the F-statistic is exceeds the critical F-value, the null hypothesis that all of the independent variables are simultaneously equal to zero should be rejected.

Reading 2: Multiple Regression

LOS 2 (i) Evaluate how well a regression model explains the dependent variable by analyzing the output of the regression equation and an ANOVA table.