Evaluation of Investment Manager Skill

Sample Evaluation

To evaluate manager performance, practitioners will use:

• Sample attribution analysis to tell us how the outperformance was achieved.
• Appraisal ratio analysis to compare Manager A's performance to other managers during the same period.
• Finally, a combination of the analyses to present a more balanced assessment of the manager's skill.

Attribution Analysis Output

The following attribution analysis summary is from the CFAI Level 3 2022 Curriculum. It shows the combination of using a sample attribution analysis combined with performance appraisal ratios to make a thorough assessment of manager performance, attributing returns to both skill and/or luck.

$$ \small{\begin{array}{c|cc|cc|ccc}
\textbf{Market} & \textbf{Manager A} & & \textbf{Benchmark} & & \textbf{Attribution} & \textbf{Effects} & & \\ \hline
& \textbf{Weight} & \textbf{5-yr} & \textbf{Weight} & \textbf{5-yr} & \textbf{Allocation} & \textbf{Selection +} & \textbf{Total} \\
& & \textbf{return} & & \textbf{return} & & \textbf{Interaction} & \\ \hline
\textbf{Japan} & 51.0\% & 12.4\% & 60.0\% & 11.5\% & -0.21\% & 0.47\% & 0.26\% \\ \hline
\textbf{Australia} & 30.0\% & 5.1\% & 25.4\% & 4.1\% & -0.24\% & 0.31\% & 0.07\% \\ \hline
\textbf{Hong Kong} & 15\% & 8.9\% & 10.0\% & 10.1\% & 0.04\% & -0.18\% & -0.14\% \\ \hline
\textbf{Singapore} & 3.5\% & 5.1\% & 3.0\% & 5.4\% & -0.02\% & -0.01\% & -0.03\% \\ \hline
\textbf{New Zealand} & 0.5\% & 8.75\% & 1.0\% & 9.1\% & 0.00\% & 0.00\% & 0.00\% \\ \hline
\textbf{Total} & \bf{100\%} & \bf{9.42\%} & \bf{100\%} & \bf{9.25\%} & \bf{-0.43\%} & \bf{0.59\%} & \bf{0.17\%}
\end{array}} $$

Key Takeaways:

• The portfolio manager lost 43 bps of performance due to allocation decisions. For example, the manager's decision to overweight Australia lost 24 bps, because Australia underperformed the total MSCI Pacific benchmark.
• The manager did not make good weighting decisions over the five-year period.
• The portfolio manager gained 59 bps of performance from stock selection decisions.
• We can conclude from the attribution analysis that the manager is a good stock picker, especially for Japanese and Australian stocks. But the manager has not been as successful in choosing the markets to allocate assets

We need to understand the risk incurred to achieve the above performance. For that risk assessment, we will consider Manager A relative to other managers, using a sample appraisal ratio analysis over the same five-year period.

Appraisal Ratios

$$ \begin{array}{c|c|c|c|c}
& \textbf{Manager A} & \textbf{Manager B} & \textbf{Manager C} & \textbf{Benchmark} \\ \hline
\text{Annualized} & 9.42 & 8.23 & 10.21 & 9.25 \\
\text{return} & & & & \\ \hline
\text{Annualized} & 10.83 & 8.10 & 12.34 & 9.76 \\
\text{Std. Dev} & & & & \\ \hline
\text{Sharpe} & 0.68 & 0.76 & 0.66 & 0.73 \\
\text{ratio} & & & & \\ \hline
\text{Treynor} & 0.35 & 0.32 & 0.19 & 0.57 \\
\text{ratio} & & & & \\ \hline
\text{Information} & 0.43 & 0.41 & 0.30 & 0.00 \\
\text{ratio} & & & & \\ \hline
\text{Sortino} & 0.82 & 0.51 & 1.03 & 0.97\\
\text{ratio} & & & & \\
\text{(MAR = 3%)} & & & & \\
\end{array} $$

Key Takeaways:

• Managers A and C have a higher volatility of returns than the benchmark.
• Manager A's Sharpe ratio is less than the benchmark's Sharpe ratio and less than Manager B's Sharpe ratio. Thus, for this period, we know Manager A certainly incurred more risk than the benchmark and Manager B did for the same amount of return generated.
• Given that Manager A has the highest Treynor and information ratios for this period, she has been able to produce a higher return relative to systematic risk.
• In addition, consider the manager's Sortino ratio of 0.82, not significantly higher than the Sharpe ratio, but again indicative of an ability to generate higher returns relative to downside risk.

Question

Which of the following statements is least likely true?

1. A manager who over weights a sector that outperforms the benchmark will exhibit a positive allocation effect.
2. A manager who over weights a sector that underperforms the benchmark will exhibit a negative allocation effect.
3. A manager who over weights a sector that underperforms the benchmark will exhibit a positive allocation effect.

Solution

The correct answer is C.

Answer choice C is incorrect, overweighting a sector that has done poorly relative to the benchmark will result in a negative allocation effect.

Answer choices A and B are both true statements.

Performance Measurement: Learning Module 1: Portfolio Performance Evaluation; Los 1(p) Evaluate the skill of an investment manager