Imagine, if you will, this not uncommon C-suite scenario:
“We have a critical decision,” the CEO says to the management team. “How much can we invest in digital to stay ahead of the disrupters?” The CFO follows: “We must forecast earnings to determine how to fund this investment.” Understanding the importance of this initiative, the department heads begin the forecasting process with their finance teams. Not wanting to “under deliver,” each department submits a slightly conservative forecast, one that they’re comfortable meeting or exceeding.
That’s known commonly as “the sandbagging problem.” And it has implications for financial institutions that instead of embracing uncertainty, fear it and hedge their bets.
Yet a new data standard coupled to improvements in Excel allows bankers to make smarter decisions. The discipline of probability management conveys uncertain forecasts as data stored in arrays called SIPs (short for “stochastic information packets”). Deployed in municipal finance, energy, defense and public utilities, probability management can also improve decisions in banking by making simulations easy, accessible and auditable.
Decision makers must bear the following in mind as they prepare to advance along the probability management path:
1. The future is uncertain. Sales projections, budgets, technology schedules, loss estimates: All of these rely on events and information we don’t know yet. As Yogi Berra said, borrowing from the Danish politician Karl Kristian Steincke, “It’s difficult to make predictions, especially about the future.”
2. Most managers—and most planning processes—forecast with a “best guess” or a single number, usually an average. “We project our profit and loss to be X.”
3. This leads to the “flaw of averages,” a set of systematic errors that arises when uncertainties are represented by single numbers. It explains why so many budgets are inflated, projects behind schedule or risks misunderstood. In short, it states that plans based on average assumptions are wrong on average.
Going back to our example, suppose the CFO says that the market expects pre-provision net revenue (PPNR) of $250 million; anything higher can be invested. The goal is $50 million, so $300 million PPNR is needed. The heads of retail, commercial, wealth, technology, marketing, risk and admin all pad their budgets by a few percent—giving them each an 80 percent chance of meeting or exceeding their numbers. (No one ever got in trouble for beating a forecast.)
The CEO and CFO also don’t want to disappoint the Board of Directors and so do not discourage an 80 percent confidence budget, which also implies a 20 percent chance of missing the target. At the company level, however, this does not translate into a 20 percent chance of missing the aggregated target, but only a 2 percent chance. How is this?
Let’s follow the math. Sandbagging overstates the chance of missing the corporate target by a factor of 10. If we planned a true 20 percent chance of missing the corporate target we would add $28 million to discretionary investment budget. Of course actual results will vary with the inputs. But remember, with more levels of management the problem gets worse—and second, you can’t do this stuff in your head!
Murphy’s flaw: When sandbagging goes to extremes
What is happening? If department managers submit an average for their budget, the enterprise will get the correct average for the aggregated results. But if they submit their 80th percentile, the enterprise will not get the 80th percentile of the aggregated financials—but, as in this case, more like the 98th percentile.
This is known as a the “flaw of extremes,” a special case of the “flaw of averages.” Think of it this way: Suppose each revenue line has an independent 50 percent chance of being exactly $1 over or under forecast. Does that mean that two revenue lines have a 50 percent chance of being a total of $2 under forecast?
It’s like getting two tails on two flips of a coin; there is only one chance in four. In fact, if you had 10 such revenue lines, the chance of being a total of $10 under forecast (the same as all 10 coin flips coming up tails) is less than one in a thousand, due to the diversification effect.
Many companies are aware of sandbagging but struggle for an easy way to address it. As a result, they suffer the consequences year after year. Today, the antidote is to embrace uncertainty with the following steps.
1. Harness those with statistical training in your organization to create SIP libraries of future scenarios. Instead of displaying a single future, the SIP approximates all possible futures with hundreds or thousands of possible outcomes.
Common techniques such as data mining, regression and simulation may be used to extract SIP libraries from historical data, and they may be stored as auditable data in any information system including spreadsheets.
2. Create interactive simulation dashboards in native Excel to assist you in choosing the appropriate degree of conservatism. This takes advantage of the powerful data table function in Excel to run thousands of simulations per keystroke.
3. Set goals and expectations in terms of the chances of meeting your goals. Do we really want everyone’s 80th percentile forecast? How much money are we leaving on the table? What is our risk tolerance for earnings? Do we need to look at our incentives if everyone is afraid of missing a forecast?
Know your “how to” list
So far, we’ve discussed one example of the “flaw of averages” in banking. There are numerous other cautions worth noting:
Aggregating operational risk across business units: how to add up tail-risks for each business to understand tail-risk for the entire organization. This is especially critical for financial services organizations, which have long risk tails (low probability of very high losses in many areas).
Setting realistic schedules for large technology projects: how uncertain timelines for each project path affect the range of completion outcomes for the entire project.
Forecasting new loan volumes and sales commission expenses: how to estimate future loan volumes and compensation costs (which are especially prone to errors resulting from the “flaw of averages”).
Weighing staffing levels for bankers, tellers and phone agents: how to manage queues and wait times to understand the costs and benefits of different service levels.
Banks that embrace uncertainty and master the discipline of probability management will stimulate better conversations about goals and risks, leading to smarter decisions. Indeed, the probability of such positive outcomes is very, very high.
Matthew Raphaelson is current chair, banking applications at ProbabilityManagement.org. Dr. Sam L. Savage, author of “The Flaw of Averages” and a consulting professor at Stanford University, is executive director of ProbabilityManagement.org.