Case Study: Model Risk and Model Valid ...
After completing this reading, you should be able to: Define a model and... Read More
After completing this reading, you should be able to:
Stress testing is a risk management tool that involves analyzing the impacts of the extreme scenarios that are unlikely but feasible. The main question for financial institutions is whether they have adequate capital and liquid assets to survive stressful times. Stress testing is done for regulatory purposes or for internal risk management by financial institutions. Stress testing can be combined with measurement of the risk such as the Vale-at-Risk (VaR) and the Expected Shortfall (ES) to give a detailed picture of the risks facing a financial institution.
This chapter deals with the internally generated stress testing scenarios, regulatory requirements of stress testing, governance issues of stress testing, and the Basel stress testing principles.
Recall that the VaR and ES are estimated from a loss distribution. VaR enables a financial institution to conclude with X% likelihood that the losses will not exceed the VaR level during time T. On the other hand, ES enables the financial institutions to conclude whether the losses exceed the VaR level during a given time T and hence the expected loss will be the ES amount.
VaR and ES are backward-looking. That is, they assume that the future and the past are the same. On the other hand, stress testing is forward-looking. It asks the question, “what if?”.
While stress testing largely does not involve probabilities, VaR and ES models are founded on probability theory. For example, a 99.9% VaR can be viewed as a 1-in-1,000 event.
The backward-looking ES and VaR consider a wide range of scenarios that are potentially good or bad to the organization. However, stress testing considers a relatively small number of scenarios that are all bad to the organization.
Specifically, for the market risk, VaR/ES analysis often takes a short period of time, such as a day, while stress testing takes relatively long periods, such as a decade.
The primary objective of stress testing is to capture the enterprise view of the risks impacting a financial institution. The scenarios used in the stress testing are often defined based on the macroeconomic variables such as the unemployment rates and GDP growth rates. The effect of these variables should be considered in all parts of an institution while considering interactions between diverse areas of an institution.
Conventional VaR and ES are calculated from data spanning from one to five years, where a daily variation of the risk factors during this period is used to compute the potential future movements.
However, in the case of the stressed VaR and stressed ES, the data is obtained from specifically stressed periods (12-month stressed period on current portfolios according to Basel rules). In other words, stressed VaR and stressed ES generates conditional distributions and conditional risk measures. As such, they are conditioned to a recurrence of a given stressed period and thus can be taken as a historical stress testing.
Though stressed VaR and stressed ES might be objectively similar, they are different. Typically the time horizon for the stressed VaR/ES is short (one to ten days), while for the stress testing, it considers relatively longer periods.
For instance, assume that a stressed period is the year 2007. The stressed VaR would conclude that if there was a repeat of 2007, then there is an X% likelihood that losses over a period of T days will not surpass the stressed VaR level. On the other hand, stressed ES would conclude that if the losses over T days do not exceed the stressed VaR level, then the expected loss is the stressed ES.
However, stress testing would ask the questions “if the following year (2008) is the same as in 2007, will the financial institution survive?” Alternatively, what if the conditions of the next year are twice as adverse as that of 2007, will the financial institution survive? Therefore, stress testing does not consider the occurrence of the worst days of 2008 but rather the impact of the whole year.
There is also a difference between conventional VaR and the stressed VaR. Conventional VaR can be back-tested while stressed VaR cannot. That is, if we can compute one-day VaR with 95% confidence, we can go back and determine how effective it would have worked in the past. We are not able to back-test the stressed VaR output and its results because it only considers the adverse conditions which are generally infrequent.
The basis of choosing a stress testing scenario is the selection of a time horizon. The time horizon should be long enough to accommodate the full analysis of the impacts of scenarios. Long time horizons are required in some situations. One-day to one-week scenarios can be considered, but three months to two-year scenarios are typically preferred.
The regulators recommend some scenarios, but in this section, we will discuss internally chosen scenarios. They include using historical scenarios, stressing key variables, and developing ad hoc scenarios that capture the current conditions of the business.
Historical scenarios are generated by the use of historical data whose all relevant variables will behave in the same manner as in the past. For instance, variables such as interest rates and the credit rate spreads are known to repeat past changes. As such, actual changes in the stressed period will be assumed to repeat themselves while proportional variations will be assumed for others. A good example of a historical scenario is the 2007-2008 US housing recession, which affected a lot of financial institutions.
In some cases, a moderately adverse scenario is made worse by multiplying variations of all risk factors by a certain amount. For instance, we could multiply what happened in the loss-making one-month period and increase the frequency of movement of all relevant risk movements by ten. As a result, the scenario becomes more severe to financial institutions. However, this approach assumes linear relationships between the movements in risk factors, which is not always the case due to correlations between the risk factors.
Other historical scenarios are based on one-day or one-week occurrences of all market risk factors. Such events include terrorist attacks (such as 9/11 terrorist attacks) and one-day massive movement of interest rates (such as on April 10, 1992, when ten-year bond yields changed by 8.7 standard deviations).
A scenario could be built by assuming that a significant change occurs in one or more key variables. Such changes include:
Some other significant variations could occur in factors such as money exchange rates, prices of commodities, and default rates.
In the case of the market risk, small changes in measured using the Greek letters (such as delta and gamma). The Greek letters cannot be used in stress testing because the changes are usually large. Moreover, Greeks are used to measure risk from a unit market variable over a short period of time, while stress testing incorporates the interaction of the different market variables over a long period of time.
The stress testing scenarios we have been discussing above are performed regularly, after which the results are used to test the stability of the financial structure of a financial institution in case of extreme conditions. However, the financial institutions need to develop ad hoc scenarios that capture the current economic conditions, specific exposures facing the firm and update analysis of potential future extreme events. The firms either generate new scenarios or modify the existing scenarios based on previous data.
An example of an event that will prompt the firms to develop an ad hoc scenario is the change in the government policy on an important aspect that impacts the financial institutions or change in Basel regulation that requires increment of the capital within short periods of time.
The boards, senior management, and economic experts use their knowledge in markets, global politics, and current global instabilities to come with adverse scenarios. The senior management carries out a brain-storming event, after which they recommend necessary actions to avoid unabsorbable risks.
While stress testing, it is vital to involve the senior management for it to be taken seriously and thus used for decision making. The stress-testing results are not only to satisfy the “what if” question, but also the Board and management should analyze the results and decide whether a certain class of risk mitigation is necessary. Stress testing makes sure that the senior management and the Board do not base their decision making on what is most likely to happen, but also consider other alternatives less likely to happen that could have a dramatic result on the firm.
It is possible to see how the majority of the relevant risk factors behave in a stressed period while building a scenario, after which the impact of the scenario on the firm is analyzed in an almost direct manner. However, scenarios generated by stressing key variables and ad hoc scenarios capture the variations of a few key risk factors or economic variables. Therefore, in order to exhaust the scenarios, it is necessary to build a model to determine how the “left out” variables are expected to behave in a stressed market condition. The variables stated in the context of the stress testing are termed as core variables, while the remaining variables are termed as peripheral variables.
One method is performing analysis, such as regression analysis, to relate the peripheral variables to the core variables. Note that the variables are based on the stressed economic conditions. Using the data of the past stressed periods is most efficient in determining appropriate relationships.
For example, in the case of the credit risk losses, data from the rating agencies, such as default rates, can be linked to an economic variable such as GDP growth rate. Afterward, general default rates expected in various stressed periods are determined. The results can be modified (scaled up or down) to determine the default rate for different loans or financial institutions. Note that the same analysis can be done to the recovery rates to determine loss rates.
Apart from the immediate impacts of a scenario, there are also knock on-effects that reflect how financial institutions respond to extreme scenarios. In its response, a financial institution can make decisions that can further worsen already extreme conditions.
For instance, during the 2005-2006 US housing price bubble, banks were concerned with the credit quality of other banks and were not ready to engage in interbank lending, which made funding costs for banks rise.
Recall that stress testing involves generating scenarios and then analyzing their effects. Reverse stress testing, as the name suggests, takes the opposite direction by trying to identify combinations of circumstances that might lead financial institutions to fail.
By using historical scenarios, a financial institution identifies past extreme conditions. Then, the bank determines the level at which the scenario has to be worse than the historical observation to cause the financial institution to fail. For instance, a financial institution might conclude that twice the 2005-2006 US housing bubble will make the financial institution to fail. However, this kind of reverse stress testing is an approximation. Typically, a financial institution will use complicated models that take into consideration correlations between different variables to make the market conditions more stressed.
Finding an appropriate combination of risk factors that lead the financial institution to fail is a challenging feat. However, an effective method is to identify some of the critical factors such as GDP growth rate, unemployment rates, and interest rate variations, then build a model that relates all other appropriate variables to these key variables. After that, possible factor combinations that can lead to failure are searched iteratively.
US, UK, and EU regulators require banks and insurance companies to perform specified stress tests. In the United States, the Federal Reserve performs stress tests of all the banks whose consolidated assets are over USD 50 billion. This type of stress test is termed as Comprehensive Capital Analysis and Review (CCAR). Under CCAR, the banks are required to consider four scenarios:
The baseline scenario is based on the average projections from the surveys of the economic predictors but does not represent the projection of the Federal Reserve.
The adverse and the severely adverse scenarios describe hypothetical sets of events which are structured to test the strength of banking organization and their resilience. Each of the above scenarios consists of the 28 variables (such as the unemployment rate, stock market prices, and interest rates) which captures domestic and international economic activity accompanied by the Board explanation on the overall economic conditions and variations in the scenarios from the past year.
Banks are required to submit a capital plan, justification of the models used, and the outcomes of their stress testing. If a bank fails to stress test due to insufficient capital, the bank is required to raise more capital while restricting the dividend payment until the capital has been raised.
Banks with consolidated assets between USD 10 million and USD 50 million are under the Dodd-Fank Act Stress Test (DFAST). The scenarios in the DFAST is similar to that in the CCAR. However, in the DFAST, banks are not required to produce a capital plan.
Therefore, through stress tests, regulators can evaluate the banks to determine their ability to extreme economic conditions consistently. However, they recommend that banks develop their scenarios.
Stress testing governance is important for determining the extent to which financial institutions do the exercise. Moreover, governance ensures that the assumption is a stress testing has been well evaluated, and the senior management analyzes the stress testing outcomes and the necessary actions taken out of relevant results.
Stress testing is a broad term used to describe a firm’s examination of its risks and the potential impact of stressful events on its financial condition. Stress testing seeks to lay bare all the financial circumstances a firm may find itself in. It helps to understand the range of potential risk exposures.
The importance of stress testing was highlighted in the post-analysis of the 2007/2008 financial crises. At many firms, stress testing was insufficiently integrated into their risk management practices. The few that had shown more commitment to stress testing did not examine sufficiently severe scenarios, and test results had little or no consideration in decision making.
Most importantly, the crisis brought to the fore the key role of governance in stress testing. In summary, the following points stand out:
The ensuing learning outcomes seek to break down these key elements even further.
For effective operation of stress testing, the Board of directors and senior management should have distinct responsibilities. What’s more, there should be some shared responsibilities, although a few roles can be set aside exclusively for one of the two groups.
A financial institution should set out clearly stated and understandable policies and procedures governing stress testing, which must be adhered to. The policies and procedures ensure that the stress testing of parts of a financial institution converges to the same point.
The policies and procedures should be able to:
A firm should ensure that its stress-testing activities are well documented, including clear and precise details about key assumptions, methodologies, limitations, uncertainties, and test results. Documentation has several advantages:
The stress testing governance covers the independent review procedures, which are expected to unbiased and provide assurance to the Board that stress testing is carried out while following the firm’s policies and procedures. Financial institutions use diverse models that are subject to independent review to make sure that they serve the intended purpose.
Areas of Validation and Independent Review for Stress Tests that Require Attention from a Governance Perspective include:
Internal audit should:
To accomplish all the above, internal audit staff must be well qualified. They should be well-grounded in stress-testing techniques and technical expertise to be able to differentiate between excellent and inappropriate practices.
The Basel Committee emphasizes that stress testing is a crucial aspect by requiring that the market risk calculations are based on the internal VaR and the Expected Shortfall (ES) models, which should be accompanied by “ rigorous and comprehensive” stress testing. Moreover, Banks that use the internal rating approach of the Basel II to calculate the credit risk capital should perform a stress test to evaluate the strength of their assumptions.
Influenced by the 2007-2008 financial crisis, the Basel Committee published the principles of the stress-testing for the banks and corresponding supervisors. The overarching emphasis of the Basel committee was the importance of stress testing in determining the amount of the capital that will cushion banks against losses due to large shocks.
Therefore, the Basel committee recognized the importance of stress testing in:
When the Basel committee considered the stress tests done before 2007-2008, they concluded that:
According to the Basel Committee on Banking Supervision’s “Stress Testing Principles” published in December 2017:
The stress testing frameworks should involve a governance structure that is clear, documented, and comprehensive. The roles and responsibilities of senior management, oversight bodies, and those concerned with stress testing operation should be clearly stated.
The stress testing framework should incorporate a collaboration of all required stakeholders and the appropriate communication to stakeholders of the stress testing methodologies, assumptions scenarios, and results.
The stress testing frameworks should satisfy the objectives that are documented and approved by the Board of an organization or any other senior governance. The objective should be able to meet the requirements and expectations of the framework of the bank and its general governance structure. The staff mandated to carry out stress testing should know the stress testing framework’s objectives.
Stress testing should reflect the material and relevant risk determined by a robust risk identification process and key variables within each scenario that is internally consistent. A narrative should be developed explaining a scenario that captures risks, and those risks that are excluded by the scenario should be described clearly and well documented.
Stress testing is typically a forward-looking risk management tool that potentially helps a bank in identifying and monitoring risk. Therefore, stress testing plays a role in the formulation and implementation of strategic and policy objectives. When using the results of stress testing, banks and authorities should comprehend crucial assumptions and limitations such as the relevance of the scenario, model risks, and risk coverage. Lastly, stress testing as a risk management tool should be done regularly in accordance with a well-developed schedule (except ad hoc stress tests). The frequency of a stress test depends on:
Stress testing frameworks should have adequate organizational structures that meet the objectives of the stress test. The governance processes should ensure that the resources for stress testing are adequate, such that these resources have relevant skill sets to implement the framework.
Stress tests identify risks and produce reliable results if the data used is accurate and complete, and available at an adequately granular level and on time. Banks and authorities should establish a sound data infrastructure which is capable of retrieving, processing, and reporting of information used in stress tests. The data infrastructure should be able to provide adequate quality information to satisfy and objectives of the stress testing framework. Moreover, structures should put place to cover any material information deficiencies.
The models and methodologies utilized in stress testing should serve the intended purpose. Therefore,
The model building should be a collaborative task between the different experts. As such, the model builders engage with stakeholders to gain knowledge on the type of risks being modeled and understand the business goals, business catalysts, risk factors, and other business information relevant to the objectives of the stress testing framework.
Periodic review and challenge of stress testing for the financial institutions and the authorities is important in improving the reliability of the stress testing results, understanding of results’ limitations, identifying the areas that need improvement and ensuring that the results are utilized in accordance with the objectives of the stress testing framework.
Communicating the stress testing results to appropriate internal and external stakeholders provides essential perspectives on risks that would be unavailable to an individual institution or authority. Furthermore, disclosure of the stress test results by banks or authorities improves the market discipline and motivate the resilience of the banking sector towards identified stress.
Banks and authorities who choose to disclose stress testing results should ensure that the method of delivery should make the results understandable while including the limitations and assumptions to which the stress test is based. Clear conveyance of stress test results prevents inappropriate conclusions on the resilience of the banks with different results.
Hardik and Simriti compare and contrast stress testing with economic capital and value at risk measures. Which of the following statements regarding differences between the two types of risk measures is most accurate?
The correct answer is C.
A is inaccurate. Stress tests tend to calculate losses from the perspective of accounting, while EC/VaR methods compute losses based on a market point of view.
B is inaccurate. While stress tests focus on conditional scenarios, EC/VaR methods focus on unconditional scenarios.
D is also inaccurate. Stress tests do not focus on probabilities. Instead, they focus on ordinal arrangements like “severe,” “more severe,” and “extremely severe.” EC/VaR methods, on the other hand, focus on cardinal probabilities. For instance, a 95% VaR loss could be interpreted as 5-in-100 events.
One of the approaches used to incorporate stress testing in VaR involves the use of stressed inputs. Which of the following statements most accurately represents a genuine disadvantage of relying on risk metrics that incorporate stressed inputs?
The correct answer is D.
The most common disadvantage of using stressed risk metrics is that they do not respond to current issues in the market. As such, significant shocks in the market can “catch the firm unawares” and result in extensive financial turmoil.
Sarah Wayne, FRM, works at Capital Bank, based in U.S. The bank owns a portfolio of corporate bonds and also has significant equity stakes in several medium-size companies across the United States. She was recently requested to head a risk management department subcommittee tasked with stress testing. The aim is to establish how well prepared the bank is for destabilizing events. Which of the following scenario analysis options would be best for the purpose at hand?
The correct answer is C.
Scenario analyses should be dynamic and forward-looking. This implies that historical scenario analysis and forward-looking hypothetical scenario analysis should be combined. Pure historical scenarios can give valuable insights into impact but can underestimate the confluence of events that are yet to occur. What’s more, historical scenario analyses are backward-looking and hence neglect recent developments (risk exposures) and current vulnerabilities of an institution. As such, scenario design should take into account both specific and systematic changes in the present and near future.
Senior management should be responsible for which of the following tasks?
The correct answer is B.
Roles II and III belong to internal audit. Role IV belongs to the board of directors.