As providers of mortgage models for financial institutions, Andrew Davidson & Co., Inc. (AD&Co) enables clients to validate their use of our models and offers documentation describing the conceptual framework of the models, back-testing results, and sample forecasts under a variety of economic conditions. We also work with analytics providers who have incorporated our models to ensure that the models works as intended.

Even with this extensive support, we often do not know how our models will be used. A model that is good for one use may not be appropriate for another. For example, valuation for hedging often is different from valuation for pricing and determining return on equity, or a model built on agency data may not be appropriate for agricultural mortgages. When we are asked or when we are provided with additional information about the client’s use, we can provide additional insight into whether the model is being used appropriately.

The determination of how a model should be employed starts with clarity on how the results of the model will be used. Note that the focus here is not on how reliable the model is or how it performs in sample or out of sample; rather, it is on what actions will be taken based upon the output of the model.

In the case of the Dodd-Frank Act Stress Test (DFAST) and the use of the stress tests to determine the Stress Capital Buffer (SCB), we know how the stress tests are being used by banking regulators and what actions are taken based upon the results of the stress test.

According to the Federal Reserve:¹ The original stress tests “played a role in bolstering confidence in the capital positions of U.S. banks during the 2007-09 financial crisis….” This, indeed, is an appropriate use for a system-wide stress test. In a time of crisis, with similar but uncertain risk throughout the financial system, a stress test may provide information about the health of the financial system and individual financial institutions that could not be determined using other measures of capital adequacy.

The Fed goes on to say that capital stress tests, “have become a critical supervisory tool” and are used to integrate “the Board's non-stress regulatory capital requirements with its stress-test-based capital requirements….” Here’s the rub. Did they become a critical supervisory tool and a basis for determining capital requirements because they were the right tool or because they were the tool that was available to the regulators to exercise discretion in setting capital requirements as they sought to replace the Advanced Approaches that utilized bank models?

Starting from basic principles, a stress test is not the best mechanism to establish capital requirements. Conceptually, capital is required to protect depositors and creditors from uncertainty. Expected losses should be built into reserves. Capital then is required for undiversified and unhedged tail risks that are borne by the financial institution. As these risks are associated with uncertainty they may not be reflected in any individual scenario. In fact, due to the availability of a wide range of financial instruments, banks can control the amount of risk in any single scenario at a modest cost and without reducing overall risk.

This creates a quandary for regulators. If they telegraph the detailed stress scenario in advance, institutions will be able to adjust their portfolios to enhance income in those scenarios, thereby reducing their required capital buffer, but not necessarily reducing risk across other potential scenarios. However, if they do not disclose the scenarios in advance, they can be (and have been) accused of being arbitrary.

The use of specific scenarios also creates issues associated with the use of models like AD&Co’s LoanDynamics Model or any model of borrower behavior within the stress test framework. Stress tests by their very nature involve scenarios that either have not occurred in the past or have been very infrequent. Moreover, no two actual stress events are the same. Thus, it is not possible to determine with precision how borrowers will behave. While models may and should provide a general indication of the performance of financial assets under stress, there may be substantial uncertainty.

Once again, the regulators face difficult choices. Should they allow each firm to develop and use their own models and recognize that there will be different results for similar assets under the same scenario at different institutions? Or should they seek consistency in results even in the face of this fundamental uncertainty? Neither solution seems quite right. Capital should reflect model risk as well as other economic uncertainties, so forcing use of a single set of modeling assumptions could increase systemic risk.

While it may seem like the current approach is beneficial if the stress scenario occurs and harmless otherwise, there are substantial costs and missed opportunities associated with the DFA Stress Tests. Firms (and the Fed) spend significant resources on the stress test because they have a direct impact on capital requirements and dividends. Those resources might be better spent on a broader set of risk measurement and risk management activities. Furthermore, stress tests may create a false impression that the banks have sufficient capital to withstand any stress or, even worse, that when stress emerges, that was not envisioned by the regulatory scenarios, such as the rate increases in 2022 and 2023, depositors and investors may have little confidence that the banks can weather the storm.

Even if the current implementation of stress tests isn’t the right approach to determining a capital buffer, can stress tests still be used to determine a capital buffer without revamping the entire capital regime?

A better approach to using stress tests would be to recognize that capital is required to bear a variety of uncertain risks. As such, other than when there is a dominant risk across the entire financial system, a variety of scenarios are required. A better approach would also recognize that interest rate risk in the banking book is not captured by current asset-based capital requirements, which focus on credit risk, scenarios which expose risk from rising and falling rates are also required. The introduction of the exploratory scenarios last year is a partial step in this direction. A better approach would also encourage financial institutions to explore the model risk associated with asset performance without penalizing firms for looking at more conservative scenarios.

This approach would involve five or possibly even ten scenarios to provide a robust evaluation of risk. 5 to 10 scenarios that are defined relative to current conditions that would stress credit, market risks, and interest rates. The scenarios could even have a counter-cyclical flavor. The Fed could choose one for the actual stress test that year (if there continues to be a requirement to have only one scenario) but with the expectation that firms would compute results and manage risk for all the scenarios since they wouldn’t know which one was going to be selected.

In this way, the stress test would operate like an exam where the professor tells you all the possible questions but only selects one or two for the final.

With a framework that includes multiple scenarios that can be consistent over time, stress tests can be a more valuable and reliable tool for determining stress capital requirements. During periods of system-wide stress, scenarios can be developed to bolster financial confidence, as during the Great Financial Crisis. In this way, scenario-based stress tests can be valuable both during and between periods of severe financial stress.

¹ “Stress Tests.” Federal Reserve Board - Stress Tests, June 22, 2022. https://www.federalreserve.gov/supervisionreg/stress-tests-capital-planning.htm.