_ wP 2oS5
POLICY RESEARCH WORKING PAPER 2085
iM onitoring Banking Sector A multivariate logit empirical
model of banking crisis
Fragility probabilities can be usefui for
monitoring fragility in the
A Multivariate Logit Approach banking sector.
with an Application to the
1996-97 Banking Crises
Aslh Demirgfii-Kunt
Enrica Detragiache
The World Bank
Development Research Group
Finance
March 1999
l PouIcY RESEARCH WORKING PAPER 2085
Summary findings
Demirguc-Kunt and Detragiache explore how a situation at individual banks or in specific segments of
miultivariate logit empirical model of banking crisis the banking sector - so crises that may develop from
probabilities can be used to monitor fragility in the specific weaknesses in some market segments and spread
banking sector. through contagion would not be detected.
The proposed approach relies on readily available The econometric study of systemic banking crises is a
data, and the fragility assessment has a clear relatively new field of study. The development and
interpretation based on in-sample statistics. Also, the evaluation of monitoring and forecasting tools based on
monitoring system can be tailored to fit the preferences the results of studies such as this are at an embryonic
of the decisionmakers, and the model has better in- stage at best.
sample performance than currently available alternatives. Demirgtii-Kunt and Detragiache highlight elements
Despite these advantages, the monitoring system that need to be evaluated in developing "ready-to-use"
would have missed the 1997 banking crises in Indonesia, procedures for decision-makers and explore possible
Malaysia, and the Republic of Korea, while it would have avenues for doing so.
derected some weakness in Thailand and the Philippines. The monitoring system must be designed to fit the
It would have clearly foreseen the 1996 crisis in Jamaica. needs of policymakers, so systems must be developed as
Aggregate variables can convey information about part of an interactive process involving both
general economic conditions often associated with econometricians and decisionmakers.
fragility in the banking sector but are silent about the
This paper - a product of Finance, Development Research Group - is part of a larger effort in the group to understand
banking crises. Copies of the paper are available free from the World Bank, 1818 H Street NW, Washington, DC 20433.
Please contact Kari Labrie, room MC3-456, telephone 202-473-1001, fax 202-522-1155, Internet address
klabrie@jworldbank.org. Policy Research Working Papers are also posted on the Web at http://wxvw.worldbank.org/html!
dec/Publications/Workpapers/home.html.The authors may be contacted at ademirguckunt@worldbank.org or
edetragiache@ imf.org. March 1999. (45 pages)
The Policy Research Working Paper Series disseminates the findings of work in progress to encourage the exchange of ideas about 1
development issues. An objective of the series is to get the findings out quickly, even if the presentations are less than fully polished. The
papers carry the names of the authors and should be cited accordingly. The findings, interpretations, and conclusions expressed in this
paper are entirely those of the authors. They do not necessarily represent the view of the World Bank, its Executive Directors, or the
countries they represent.
Monitoring Banking Sector Fragility: A Multivariate Logit Approach with
an Application to the 1996-97 Banking Crises
by Aslh Demirgii9-Kunt and Enrica Detragiache*
JEL Classification Numbers: E44, G21, C53
Keywords: Banking sector fragility, monitoring, Asian crises.
* Principal Economist, Development Research Group, The World Bank, Room N-903 5, 1818 H
Street, Washington, DC, 20433, and Economist, Research Department, International Monetary Fund,
Room 9-71 8H, 700 19th Street, NW, Washington, DC 20431, respectively. We wish to thank
Anqing Shi for capable research assistance.
1
1. Introduction
The last two decades have seen a proliferation of systemic banking crises, as
documented, among others, by the comprehensive studies of Lindgren, Garcia, and Saal
(1996) and Caprio and Klingebiel (1996). Most recently, the economic crises experienced by
five East Asian countries (Indonesia, Malaysia, South Korea, the Philippines, and Thailand)
were accompanied by deep financial sector problems. While in some cases the troubles were
foreseen, in others most observers (including the Fund, the Bank, and the major credit rating
agencies) were caught by surprise. Similarly, three years earlier, the Mexican devaluation
and associated banking crisis had caught many observers and market participants by surprise.
The spread of banking sector problems and the difficulty of anticipating their
outbreak have raised the issue of improving monitoring capabilities both at the national and
supra-national level, and, particularly, of using statistical studies of past banking crises to
develop a set of indicators of the likelihood of future problems. In our previous work
(Demirguic-Kunt and Detragiache, 1998a and 1998b), we developed an empirical model of
the determinants of systemic banking crises for a large panel of countries. Using a
multivariate logit framework, we estimated the probability of a banking crisis as a function of
various explanatory variables. That research showed that there is a group of variables,
including macroeconomic variables, characteristics of the banking sector, and structural
characteristics of the country, that are robustly correlated with the emergence of banking
sector crises. This paper explores how the information contained in that empirical
relationship can be utilized to monitor banking sector fragility.
2
The basic idea is to estimate a specification of the multivariate logit model used in our
previous work that relies only on explanatory variables whose future values are routinely
forecasted by professional forecasters, the Fund, or the Bank. Out-of-sample banking crisis
probabilities are then computed using the estimated coefficients and forecasted values of the
explanatory variables. Using the information provided by in-sarnple estimation result, these
forecasted probability are used to make a quantitative assessment of fragility. More
specifically, we examine two different monitoring frameworks: in the first, the monitor wants
to know whether forecasted probabilities are high enough to trigger a response or not. The
response is defined to be a costly action of some sort, for instance gathering new specific
information, scheduling on-site bank inspections, taking preventive regulatory measures, or
others. Each possible threshold for taking action has a cost in terms of type I error (failure to
identify a crisis) and type II error (false alarm), a cost that can be quantified on the basis of
in-sample classification accuracy. Naturally, the choice of the criterion depends on the cost of
either type of error to the monitor. For instance, if the monitoring system is used as a
preliminary screen to determine which cases warrant further analysis, then a system that
tolerates a fair amount of type II errors but incurs few type I errors will be preferable to one
that is likely to miss a lot of crises. Conversely, if the "warning system" is used to put
pressure on country authorities to take drastic policy actions to prevent an impending
disaster, then a more conservative criterion is desirable. The framework developed here will
be sufficiently flexible to accommodate alternative preferences for the decision-maker, and it
will make explicit the costs and benefits of alternative criteria.
In the second monitoring framework examined, the monitor is simply interested in
rating the fragility of the banking system. Depending on the rating, various courses of action
3
may follow, but these are not explicitly modeled. In this case, it is desirable for a rating to
have a clear interpretation in terms of crisis probability, so that different ratings can be
compared. We examine one such example.
As an illustration of the monitoring procedures developed in the first part of the
paper, in the second part of the paper we compute out-of-sample banking crisis probabilities
for the six banking crises that occurred in 1996-97, namely Jamaica in 1996 and the five East
Asian crises in 1997. To compute these probabilities, we use forecasts produced in the first
part of the crisis year by professional forecasters (as reported in Consensus Forecasts) and by
the Fund through its semi-annual World Economic Outlook (WEO) exercise. ' For the Asian
countries, these forecasts do not predict the large exchange rate depreciations that occurred in
the second half of the year. The exercise suggests that failure to predict the exchange rate
collapse and its immediate consequences would have led a monitor using our system to
conclude that there was no little of banking sector fragility in Indonesia, Korea, and
Malaysia, while some signs of weakness would have been detected for Thailand and the
Philippines. The Jamaican crisis would have been identified quite clearly.
The paper is organized as follows: the next section will briefly review existing
literature on banking system fragility indicators; Section III presents an adapted version of
our empirical model of banking crises. Section IV discusses how out-of-sample probability
1 Our strategy was to use WEO forecasts only where variables were not available
from Consensus Forecasts. The latter forecasts are computed as averages of forecasts by
several market participants. Thus, they should better represent the view prevailing on the
market about economic conditions in East Asia than the WEO, which reflects the assessment
of the Fund alone.
4
forecasts obtained from the model can be used to obtain an early warning system. Section V
contains an application to the crises of 1996-1997, while Section VI concludes.
II. The Literature
An extensive literature has reviewed episodes of banking crises around the world,
examining the developments leading up to the crisis as well as the policy response. This
work, while it does not directly address the issue of leading indicators of banking sector
problems, points to a number of variables that display "anomalous" behavior in the period
preceding the crises. For instance, Gavin and Hausman (1996) and Sachs, Tornell, and
Velasco (1996) suggest that credit growth be used as an indicator of impending troubles, as
crises tend to be preceded by lending booms. Mishkin (1994) highlights equity price
declines, while, in his analysis of Mexico's 1995 crisis, Calvo (1996) suggests that
monitoring the ratio of broad money to foreign exchange reserves may be useful in
evaluating banking sector vulnerability to a currency crisis.
Honohan (1997) performs a more systematic evaluation of alternative indicators: he
uses a sample of 18 countries that experienced banking crises and 6 that did not. The crisis
countries are then divided into three groups (of equal size) according to the type of crisis
(macroeconomic, microeconomic, or related to the behavior of the government). The average
value of seven alternative indicators for the crisis countries is then compared with the average
for the control group of countries. This exercise shows that banking crises associated with
macroeconomic problems were characterized by a higher loan-to-deposit ratio, a higher
foreign borrowing-to-deposit ratio, and higher growth rate of credit. Also, a high level of
lending to the government and of central bank lending to the banking system were associated
5
with crises related to government intervention. On the other hand, banking crises deemed to
be of microeconomic origin did not appear to be associated with abnormal behavior on the
part of the indicators examined in the study.
Rojas-Suarez (1998) proposes an approach based on bank level indicators, similar in
spirit to the CAMEL system used by U.S. regulators to identify problem banks. The author
argues that in emerging markets (particularly in Latin America), CAMEL indicators are not
good signals of bank strength, and that more information can be obtained by monitoring the
deposit interest rate, the spread between the lending and deposit rate, the rate of credit
growth, and the growth of interbank debt. Because these variables are to be measured against
the banking system average, however, this approach appears more adequate for identifying
weaknesses specific to individual banks rather than a situation of systemic fragility. Also, the
approach requires bank level information, which is often not readily available outside of
developed countries.
The most comprehensive effort to date to develop a set of early warning indicators for
banking crises (and for currency crises) is that of Kaminsky and Reinhart (1996),
subsequently refined in Kaminsky (1998). These studies examine the behavior of 15
macroeconomic indicators for a sample of 20 countries which experienced banking crises
during 1970-1995.2 The behavior of each indicator in the 24 months prior to the crisis is
contrasted with the behavior during "tranquil" times. A variable is deemed to signal a crisis if
at any time it crosses a particular threshold. If the signal is followed by a crisis within the
next 24 months, then it is considered correct; otherwise it is considered noise. The threshold
2 For a study of early warning indicators of currency crises, see also IMF (1998).
6
for each variable is chosen to minimize the in-sample noise-to-signal ratio. The authors then
compare the performance of alternative indicators based on the associated type I and type II
errors, on the noise-to-signal ratio, and on the probability of a crisis occurring conditional on
a signal being issued.3 The indicator with the lowest noise-to-signal ratio and the highest
probability of crisis conditional on the signal is the real exchange rate, followed by equity
prices and the money multiplier. These three indicators, however, have a large incidence of
type I error, as they fail to issue a signal in 73-79 percent of the observations during the 24
months preceding a crisis. The incidence of type II error, on the other hand, is much lower,
ranging between 8 and 9 percent. The variable with the lowest type I error is the real interest
rate, which signals in 30 percent of the pre-crisis observations. The high incidence of type I
error relative to type II error may not be a desirable feature of a warning system if the costs of
false alarms are small relatively to the costs of missing a crisis.
The approach chosen by Karninsky and Reinhart (1996) offers no systematic way of
combining the information contained in the various indicators, as these indicators are
examined one at a time. Presumably, the likelihood of a crisis is greater when several
indicators are signaling at the same time; on the other hand, if signals are conflicting, it is not
clear which of the indicators should be relied upon. Kaminsky (1998) addresses these
concerns by developing "composite" indexes, such as the number of indicators that cross the
threshold at any given time, or a weighted variant of that index, where each indicator is
weighted by its signal-to-noise ratio, so that more informative indicators receive more
3 Actually, the authors use an "adjusted"version of the noise-to-signal ratio, computed
as the ratio of the probability of type II error to one minus the probability of a type I error.
7
weight. Preliminary results, however, indicate that the composite indicators perform worse
than the individual ones.
The approach developed in the following sections will allow the policy-maker to
choose a warning system that reflects the relative cost of type I and type II error, and it will
offer a natural way of combining the effect of various economic forces on banking sector
vulnerability. By making better use of all available information, the system will deliver lower
overall in-sample forecasting errors than those associated with individual indicators. Also, we
will examine a problem that is not addressed by Kaminsky and Reinhart, namely that of a
monitor who wishes to use informnation contained in the statistical analysis of past crisis
episodes not just to "call" or "not call" a crisis, but to obtain a more nuanced assessment of
banking sector fragility.
III. Estimating In-Sample Banking Crisis Probabilities in a Multivariate Logit
Framework
The starting point of our analysis is an econometric model of the probability of a
systemic banking crisis. In previous work (Demirguic-Kunt and Detragiache, 1 998a and
1 998b), we have estimated various alternative specifications of a logit regression for a large
sample of developing and developed countries, including both countries that experienced
banking crises and countries that did not. Details on sample selection, the construction of the
banking crisis variable, and the choice of explanatory variables can be found in our previous
papers. That work identifies a set of variables, including macroeconomic variables,
characteristics of the banking sector, and structural characteristics of the economy, that are
robustly correlated with the emergence of systemic banking crises.
8
To form the basis of an easy-to-use monitoring system, the econometric model should
rely on data that is readily available, and, if it utilizes contemporaneous values of the
explanatory variables, it should use variables for which forecasts are routinely produced.
Accordingly, in this section we will present estimation results for a specification of our
empirical model that includes only variables available from the International Financial
Statistics or other publicly available data bases, and that are routinely forecasted by the Fund
in its bi-annual World Economic Outlook (WEO) exercise, or by professional forecasters as
reported by Consensus Forecasts. As it turns out, this is not the specification that fits the data
the best. There is one important exception to this criterion, however: professional forecasts
for the nominal interest rate are not available for most countries in the sample, but excluding
the real interest rate from the regression would have meant losing a lot of explanatory power.
Thus, we decided to include the real interest rate in the regression, and use the real interest
rate at t- 1 as a "naive" forecast of the real interest rate at t in the out-of-sample forecast
exercise. The regression is estimated using a panel of 766 observations for 65 countries
during 1980-95.4 In this panel, 36 systemic banking crises were identified, so that crisis
observations make up 4.7 percent of the sample. Table 1 lists the crisis episodes. The set of
explanatory variables capturing macroeconomic conditions includes the rate of growth of real
GDP, the change in the terms of trade, the rate of depreciation of the exchange rate (relative
to the US$), the rate of inflation and the fiscal surplus as a share of GDP. The explanatory
variables capturing characteristics of the financial sector are the ratio of broad money to
4 Due to lack of data or breaks in the series, for some countries part of the sample
period may be excluded. Also, years in which banking crises are ongoing are excluded from
the sample.
9
foreign exchange reserves and the rate of growth of bank credit lagged by two periods.
Finally, GDP per capita is used as a proxy of structural characteristics of the economy that
may be relevant to the well-functioning of the banking system, such as the quality of
prudential regulation and supervision, or the enforcement of laws and regulations.
The estimated coefficients of the logit regression are reported in Table 2. Low GDP
growth, a high real interest rate, high inflation, strong growth of bank credit in the past, and a
large ratio of broad money to reserves are all associated with a high probability of a banking
crisis. Exchange rate depreciation and the terms of trade variable, on the other hand, are not
significant. Interestingly, a large fiscal surplus appears to increase the probability of a
banking crisis (although the significance level is low), suggesting that moral hazard may be
more severe when the government's comfortable financial position may make a generous
bailout more likely (Dooley, 1996).
As an illustration of the results, Table 1 shows the estimated crisis probabilities for
the 36 episodes included in the sarnple. The probabilities range from a low of 1.1 percent for
Nigeria to a high of 99.9 percent for Israel. About 70 percent of the episodes have an
estimated probability of 4 percent or above, while only 17 percent have an estimated
probability of over 50 percent. Each crisis probability can also be broken down into its
various components, so that it is possible to understand which factors are contributing to
fragility, at least according to the model, in each particular episode.
The 1994 Mexican Crisis according to the empirical model
As an illustration, Figure 1 shows the estimated crisis probability in Mexico in the
period leading up to the 1994 crisis. The probability exhibits a sharp increase in 1993, so it is
10
particularly interesting to examine which factors account for such an increase. Table 3
provides a breakdown of the factors that make up the crisis probability in 1992 and 1993
according to the econometric model. The estimated crisis probability in the two years is
reported in the last two rows of the table. The first column gives the percent change in each
explanatory variable between 1992 and 1993. The next two columns report the "weights"
given to each factor in 1993 and 1992, respectively. These weights are obtained by
multiplying the estimated regression coefficient of each variable with the corresponding
value of the variable. Negative weights indicate that the variable in question tended to
decrease the estimated crisis probability. The weights are useful in understanding the role
each factor plays in determining the overall crisis probability in a particular year; for
example, in 1993, high past credit growth, high real interest rates, and high inflation were the
main underlying reasons why the crisis probability was high in Mexico.
Of course, the factors that explain the level of crisis probability are not necessarily the
same factors that explain the change in this probability. Therefore, the table also reports
change in factor weights between 1992 and 1993, and the corresponding change in crisis
probability. Because logit is non-linear the sum of the contribution of each variable does not
always add up to the total change in probability. Looking at macro factors, one sees that
Mexico had a negative growth shock which increased the crisis probability significantly.
There was also a significant increase in real interest rates and a minor terms of trade shock.
At the same time, appreciation of the exchange rate, a lower inflation and a lower budget
surplus helped offset some of this increase. Financial sector variables played a less important
role in explaining the overall increase in probability, slightly offsetting the impact of the
macro factors. Vulnerability of the financial system to capital outflows - measured by
11
M2/reserves ratio - decreased slightly, leading to a 1 percent decrease in crisis probability.
Credit growth slowed down, leading to a 2 percent lower crisis probability. Finally, GDP per
capita- which we use as a proxy of institutional development- did not change significantly in
this period. Thus, decomposing the crisis probability helps understanding which factors
played a role in bringing about the crisis, at least according to the empirical model.
Out-of-sample probability forecasts
Because the purpose of monitoring is to obtain an assessment of future fragility, the
next step is to obtain forecasts of banking crisis probabilities. These can be easily obtained as
follows: let P be a 1 xN vector containing the N estimated coefficients of the logit regression
reported in Table 1, and let 7, be a Nx 1 vector of out-of-sample values of the explanatory
variables for country i at date t. Of course, these values can be true forecasts, estimates of
past values, or simply data for countries/time periods not included in the sample. Then, the
out-of-sample probability of a banking crisis for country i at date t is
exp[z.t]
it 1 +exp[pz,j]
Once out-of-sample probabilities are computed, the question arises of how to interpret them:
is a 10 percent crisis probability high or low? Should a policy-maker undertake preventive
actions when faced with such a probability? Should a surveillance agency issue a warning?
The next section will address the issue of how to use the forecasted probabilities to monitor
banking sector fragility.
12
IV. Building An Early Warning System Using Estimated Crisis Probabilities
The first monitoring framework considered is one in which, after forecast
probabilities are obtained as described in the preceding section, the decision-maker has to
choose whether the probability is large enough to issue a warning. This is the framework
implicit in Kaminsky and Reinhart (1996). Issuing a warning will lead to some type of action,
perhaps investment in further information gathering, such as the acquisition of bank-level
balance sheet data, or discussions with senior bank managers, bank supervisory agencies in
the country, or other market participants. Alternatively, the warning may be thought of as
leading to preventive policy or regulatory measures. Naturally, the premise behind the
warning system is that either banking crises can be prevented or that their cost can be
substantially reduced if an accurate advance warning is received by the decision-maker.
The choice of the threshold for issuing a warning will generally depend on three
aspects: first, the probability of type I and type II errors associated with the threshold, which,
assuming that the sample of past crises is representative of future crises, can be assessed on
the basis of the in-sample frequency of the two errors. Clearly, the higher the threshold that
forecasted probabilities must cross before a warning is issued, the higher will be the
probability of a type I error and the lower will be the probability of a type II error, and vice
versa. The second parameter on which the choice of the threshold depends is the
unconditional probability of a banking crisis, which can also be assessed based on the in-
sample frequency of crisis observations: if crises tend to be rare events, then the overall
likelihood of making a type I error is relatively small, and vice versa. Finally, the third aspect
that affects the choice of a warning threshold is the cost to the decision-maker of taking
preventive action relative to the cost of an unanticipated banking crisis.
13
A Loss Function for the Decision-Maker
Based on the above considerations, a more formal analysis of the decision process
behind the choice of a warning system may be stated as follows. Let T be the threshold
chosen by the decision-maker, so that if the forecasted probability of a crisis for country i at
time t exceeds T, then the system will issue a warning. Let p(T) denote the probability that
the system will issue a warning, and let e(T) be the joint probability that a crisis will occur
and the system issues no warning. Further, let cl be the cost of taking preventive actions as a
result of having received a warning signal, and let c2 be the additional cost of a banking crisis
if it is not anticipated (if anticipating a crisis can prevent it altogether, than c2 is the entire
cost of the crisis). Presumably, cl is substantially smaller than c2 if further information
gathering can be relied upon to provide useful information, and if the knowledge that a crisis
is impending allows policy-makers to take effective preventive measures. Then, a simple
linear expected loss function for the decision-maker may be defined as follows:
L(T)Ep(Y)c, +e(T)c2.
This expression can be rewritten using the notions of type I and type II error. Let a(T) be the
type I error associated with threshold T (the probability of not receiving any warning
conditional on a crisis occunring), and let b(T) be the probability of a type II error (the
probability of receiving a warning conditional on no crisis taking place). Also, let w denote
the (unconditional) probability of a crisis. Then the loss function of the decision-maker can
be rewritten as
14
L(T) =c [(1 -a(T))w +b(T)(l -w)] +c2a(T)w =wc [ 1 +( c2-c)a(T) +b( l-
Cl W
The second part of the equality above shows that the higher is the cost of missing a crisis
relative to the cost of taking preventive action (the larger is c2 relative to cl), the more
concerned the decision-maker will be about type I error relative to type II error, and vice
versa. Also, the higher is the unconditional probability of a banking crises (measured by the
parameter w), the more weight the decision-maker will place on type II errors, as the
frequency of false alarms is greater when crises tend to be rare events. Notice also that
minimizing the noise-to-signal ratio (in our notation, b(T)/(1 - a(T)) -- the criterion chosen by
Kaminsky and Reinhart to construct and rank alternative signals -- does not generally lead to
minimizing the expected loss function specified above.
Using in-sample frequencies as estimates of the true parameters, the parameter w
should be equal to the frequency of banking crises in the sample, namely 0.047. The
functions a(T) and b(T), that trace how error probabilities change with the threshold for
issuing warnings, can be obtained from the in-sample estimation results as follows: given a
threshold of -- say -- T = 0.05, we can obtain a(0.05), i.e. the associated probability of type I
error, as the percentage of banking crises in the sample with an estimated crisis probability
below 0.05. Similarly, b(O.05), the probability of issuing a warning when no crisis occurs, is
the percentage of non-crisis observations with an estimated probability of crisis above 0.05.
Figure 2 shows the functions a(T) and b(T) for T E [0, 1] computed from the estimation
results of Section III above. Of course, a(T) is increasing, as the probability of not issuing a
warning when a crisis occurs increases as the threshold rises, while b(T) is decreasing. The
15
two functions cross at T = 0.036, where the probabilities of either type of error is about 30
percent.
Figure 2 also shows that crisis probabilities estimated through our multivariate logit
framework can provide a more accurate basis for an early warning system than the indicators
developed by Kaminsky and Reinhart (1996): as discussed in section II above, the indicator
of banking crises associated with the lowest type I error in the Kaminsky-Reinhart
framework is the real interest rate, with a type I error of 70 percent and a type II error of 19
percent. With our model, as shown in Figure 2, a threshold for type I error of slightly over 70
percent (72 percent, to be precise) comes at the cost of a type II error of only 1.2 percent.
Similarly, the best indicator of banking crises according to Kamninsky and Reinhart is the real
exchange rate, with a type I error of 73 percent and a type II error of 8 percent (resulting in an
adjusted noise-to-signal ratio of 0.30). With our model, a type II error of 7.4 percent can be
obtained by choosing a probability threshold of 0.09, and it is associated with a type I error of
only 53 percent, resulting in an adjusted noise-to-signal ratio of 0.25. We conjecture that the
better performance of the multivariate logit model stems from its ability to combine into one
number (the estimated crisis probability) all the information provided by the various
economic variables monitored.
Choosing the Optimal Threshold
By way of illustration, we have computed loss functions for three alternative
configurations of the cost paramneters of the decision-maker. The cost of taking further action
as a result of a warning cl is normalized to 1 in all three scenarios, while the cost of suffering
an unanticipated crisis C2 takes the values 20, 10, and 5 respectively. The three resulting loss
16
functions are plotted in Figure 3V The values of the warning threshold that minimize the loss
functions are, respectively, T=0.034, T=0.09, and T=0.20. In other words, a decision-maker
whose cost of missing a crisis is 10 times the cost of taking precautionary measures would
issue an alarm every time the forecasted probability of crisis exceeds 9 percent, and similarly
for the other cases. Thus, as expected, as the cost of missing a crisis increases relatively to the
cost of taking preventive action, the optimal threshold of the warning system falls, resulting
in a warning system with fewer type I errors and more type II errors.
Figure 4 shows the optimal probability threshold for a broad range of values of the
parameter c2, namely c2 E [2, 40], while cl is kept constant at 1. For values of c2 between 40
and 15 the optimal probability threshold for issuing a warning is T = 0.034. With this
criterion, the probability of not issuing a warning when a crisis occurs is about 14 percent,
while the probability of mistakenly issuing a warning is 31 percent. As c2 declines below 15,
the threshold increases to 0.09 (type I error of 50 percent, and type II error of 7.4 percent),
and remains there until C2 reaches 8. At this point, the threshold jumps to 0.20, as the
decision-makers is very concerned about false alarms. Finally, if the cost of missing a crisis
is as low as 2-3 times that of issuing a false warning, then the optimal threshold is 0.30,
corresponding to a type I error as high as 72.2 percent and a type II error as low as 1.2
percent.
To fully appreciate the nature of the warning system, it is worth pointing out that the
probability of a type I error is not the probability of missing a crisis. To obtain the probability
of missing a crisis, the probability of a type I error must be multiplied by the unconditional
5 To keep the image sufficiently clear in the relevant range, we have omitted values of
the loss functions for T > 0.30. The functions continue to increase in the omitted range.
17
probability of a crisis, which in our sample is 0.047. Similarly, the probability of issuing a
wrong warning is the size of the type II error times the frequency of non-crisis observations.
With a threshold of T=0.09, the probability of missing a crisis is, therefore, only 2.3 percent,
since crises occur rarely. In contrast, the probability of receiving a false alarm is 7.1 percent,
because non-crisis observations tend to be the majority.
So, based on our framework for forecasting crisis probabilities, warning systems
associated with a relatively low incidence of type I error (below 15 percent) give rise to a
fairly large amount of false alarms, in part because crises tend to be infrequent events. If the
system is used as a preliminary screen, and further information gathering can provide a an
effective way to sort out cases in which the banking system is sufficiently sound, then the
decision-maker would be willing to accept the high incidence of type II error. It should also
be pointed out that, in some cases, what is considered a false alarm by the model may
actually be a useful signal. To illustrate this point, we have examined the "false
alarms"generated in-sample by a threshold of 0.047. As it turns out, in 21 cases the "false
positives" were observations in the two years immediately preceding a crisis, suggesting that
the conditions that eventually led to a full-fledged crisis were in place (and were detectable) a
few years in advance. In other cases, the "false alarms" may have corresponded to episodes
of fragility that were not sufficiently severe to be classified as full-fledged crises in our
empirical study, or where a crisis was prevented by a prompt policy response. Thus, an
assessment of the accuracy of the warning system based on in-sample classification accuracy
may exaggerate the incidence of type II errors. On the other hand, as usual, out-of-sample
predictions are subject to additional sources of error relative to in-sample prediction: the
forecasted values of the explanatory variables include forecast errors, and there may be
18
structural breaks in the relationship between banking sector fragility and the explanatory
variables which make predictions based on past behavior inadequate. Also, despite the large
size of our panel, the number of systemic banking crises in the sample (36) is still relatively
small, so that small sample problems may affect the estimation results. Obviously, as more
data become available and the size of the panel is extended, this problem should become less
severe.
V. Using Estimated Crisis Probabilities to Construct a Rating System for Bank
Fragility
In this section, we consider the problem of a monitor whose task is to rate the fragility
of a given banking system. The rating will then be used by other agents to decide on a
possible policy response, but the monitor is not necessarily aware of the costs and benefits of
such policy actions. In this case, it seems desirable for the rating system to have a clear
interpretation in terms of type I and type II error. This has two advantages: first, agents who
learn the rating can do their own cost/benefit calculations when they decide whether or not to
take action; second, the fragility of two systems that are assigned two different ratings can be
compared based on a clear metric.
The starting point for constructing the rating system is once again the set of forecasted
crisis probabilities obtained using the coefficients estimated in the multivariate logit
regression of Section III above. Clearly, a country with a forecasted probability of x should
be deemed more fragile than one with an estimated probability of y