The incapable soothsayers

Having a sense to “see what’s around the corner” of the economy would be an important trait for business people to have, though it’s always too tall an order to accomplish.  Instead of accepting record low volatility, which somewhat implies a drainage of uncertainty from forecasting future paths, the question we need to critically ask ourselves instead is can it even be possible for any business person (including CEOs and his or her economic advisers) to actually “see what’s around the corner” of the economy?  The answer is no.  Capital stakeholders need to instead know other skills such as when and how to set prices, invest in new labor or production capacity or risk management activities, and launch a new product.  And all of these skills, which don't need to be compensated as disproportionately from everyone else, must all come from other business instincts.

In this article we explore one high-level, macroeconomic data variable: unemployment (UR).  These statistics patterns are generic, and so the learnings from the Federal Reserve’s (FR’s) forecast of UR can be generalized to common private sector forecasting of other inter-related macro variables.  We saw through the Administration’s $700b TARP bailout program, that business leaders lacked having even such a small amount of insight into the macroeconomy when it mattered the most.  And then it would have been extraordinarily valuable in understanding the basic growth and risk opportunities in a number of vital U.S. industries, as the very survival of firms came down to a quick utilization of the capital markets over just that year.  

We have freely available the FR data for this study covering 25 years: 1988, through 2012.  The reference of each UR forecast was the following year’s Q4 average.  In 3 of those 25 years, the Federal Open Market Committee (FOMC) provided their forecast a month prior in November - instead of December.  Any Wall Street economist or trader would concede that the FR forecasts are indistinguishable from those of other private sector economists, as shown also on the topmost link.  This makes sense given a revolving door of talent.  

Look at the raw UR data, in blue below, and the FR forecast for those same reference times in green.  In 2012, the FR had forecasted 2013 UR of 7.6%, but the actual UR was better at 7.0%.  This difference shown in red is an overshoot of 0.6% (7.6%-7.0%).  This one-year forecast difference is a lot, noticing as a comparison that the entire UR rise after the 1990s internet bubble crash was just over 2.0%.  

  

      

The level of the FR’s forecast seems to fit though, as the blue and green lines appear to overlap.  Their individual standard deviations are about 1.5%, though the deviations including later revisions for the red differences are much lower at 0.7% (roughly the same as the FOMC’s calculation on published results, though this is later misrepresented with isolated discussions of normalized the data).  Does 0.7%<1.5% prove that business investments in macroeconomic forecasting are providing extraordinary value?  Hardly.  Look again at the blue and green lines.  Notice that as you follow the lines across time, that the gap between the lines increases?  The green line is just “reacting to”, instead of “forecasting in advance”!  There is zero value in rearticulating the past, and suggesting that it is the future.  As an analogy, a farmer concerned about an imminent storm doesn’t record the day’s weather forecast, only to watch it a month later. 

There is another issue with the 0.7% difference.  Organizations are investing an extraordinary amount in technology, advanced economic research, and refined mathematics modeling, with the hope that it would yield much better forecast results over time.  But this hasn’t worked in an important sense.  Consider the increasingly voluminous FOMC reports (and their high-level stream of economic and analytical notes prepared by teams for them), just to support an interest rate decision.  

Over the first half of the time series shown, the Green Book reports have grown from 30 pages, to 50 pages.  Extra data here only appears to offer exceptional insight, but it instead disarms us with a false sense of having all-knowing wisdom.  What could possibly go wrong?  Well, while the report pages have gone up, their forecast accuracy has plummeted.  A disheartening pattern that continues through the Great Recession, and on through today.  During all of 2008, we could notice for example that the head of asset management at HSBC (the world’s largest bank) about the markets argued the entire year, through the added lens of “personal wisdom”, that we were not in an alarming recession but rather a “slowdown”.  As Ronald Regan said on the 1980s campaign trail, “(a) recession is when your neighbor loses his job.  A depression is when you lose yours.”  Through the eyes of a well-heeled executive, the same depressing macroeconomic data just looks not as bad.

To see these forecasting differences up close, let’s look at the absolute level of the FR forecast difference, ignoring the direction of the red bar and just focusing on the overall amplitude of error.  The red dashed line in the chart shows how these differences have grown (not gotten better), over the past 25 years.  In the late 1980s, the typical error was just +0.2%, and it has grown with an annual rate of about 0.02%.  So after 25 years in the time series, this comes to an absolute error rate of 0.7% (0.2%+0.02%*25).  

We should clarify that while the value is the same, the 0.7% is difference from the overall standard deviation of the red bars of 0.7%.  The latter is based on the entire 25-year history, but since the standard error calculation is biased towards the higher absolute amplitudes (given that it is weighted on the difference²), both difference values come to about 0.7%.

There are no major statistical issues in directly contrasting the collection of 25-years of FR forecasts versus the actuals.  They both exhibit possible heteroskedacity, or changing variances across time, yet we consider proportional econometric data to be fairly range-bound and stable.  These ideas are more than confirmed by leading macroeconomists Sims, and Sargent, and Tinbergen.  Statisticians would fairly counter the idea though that we have identical and independent distribution (i.i.d.) changes.  For example, for the UR data set, if we had a normal haphazard guesser who general forecasted correctly, but also experienced a 1.5% variation in estimate, would have red bars for the differences deviate roughly 1.5%*2~2.2%.  So only in that sense does the 0.7% standard difference appear fine.  But we also know something is awry since there is a grossly lagging nature, between the blue line and the green line that is reacting to it.  

True forecasting focuses on creating significant forecasting insight over time, and beyond what the obvious.  For example say the population grows at 1% annual rate (+0.1% standard deviation).  A businessperson can estimate, for both of the next two years, that the population would instead grow 0.7%.  Two years later we see that the population grew 0.9% in both years.  The executive doesn’t look good for either just guessing positive numbers, or suggesting they are 80% correct (e.g., 0.7%/0.9%).  Any dummy could have defaulted to the 1% estimate, with its 0.1% standard error.  Instead the business executive has gone out of their way to perform a disservice by taking risks and doubling the typical error of 0.1%.  His or her error is now 0.2% (0.7% instead of 0.9%).

Another obvious aspect about the macroeconomic forecasts, in addition to trend, is that there is auto-correlative (serial correlative) cycles in economic data.  Serial correlation is a pattern where a move above or below the average generally stays in the same direction (i.e., above of below average) during the subsequent period.  Look again at the chart above.  And notice the pattern of three cycles?  All three UR peaks on the chart of course coincide with the official NBER troughs of about 1991, 2001, 2009.  The two intervening valleys also fit with when the growth peaked, and before the economy subsequently slipped into a recession.  These large business cycles are important and glaring parts of the trend, as statistically relevant as the 25-year UR average itself.  Over time market participants do not reward CEOs for these common sense serial correlation in the macroeconomy, and then passing those insights off as i.i.d.  Looking again at the red difference bars in the chart, and one will see that the direction of the differences tends to sustain for multiple periods.  Sometimes the differences sustain above 0% for many years, other times it is below 0%.  The resulting annual red, difference bars, run in both positive and negative streaks.

The proper benchmark metric to evaluate a CEO’s ability to forecast macroeconomic trends for their organization could start with a baseline of the change in forecasts for a given year.  Then evaluate this versus the change in the actual data for the same reference period:

(FR₁-FR₀) - (Actual₁-Actual₀) = (FR₁-Actual₁) - (FR₀-Actual₀)

In plain English, we can state this as the difference in changes (left side of the equation), or in the later charts we can state the change in differences (right side of the equation).  While less intuitive, the latter expression is mathematically equivalent to the former.  This change concept is important as it more closely weeds out the obvious statistical patterns in the green and blue time series.

The change in annual UR has a standard deviation is 0.9% (for actual, or forecast).  While the differences in change, between forecast and actual, have the same standard deviation of about 0.7%.  Except here the 0.7% deviations are no longer attractive relative to the 0.9% change deviations we noted two sentences ago, as it does against the 1.5% difference deviations we saw early in the article.

The chart below shows the new results. For illustration, from 2012 to 2013, the actual UR fell about 7.8%, to the 7.0% we label on the top chart.  This drop of 0.9% is shown on the blue line below (it’s mathematical convention to use an apostrophe to the variable name to represent change).  Also in 2013, the FR forecasted UR improved from about 8.6%, to the 7.6% we label on the top chart.  This drop of 1.1% is shown on the green line below.  The change here is that the FOMC overshot the UR improvement by 0.2% (-1.1% instead of -0.9%).

We show the red error bars are now more significant and random.  The green line more clearly lags the blue line.  Further, the serial correlation has disappeared.  Using the Pearson approximation to the Durbin-Watson method, we have a quick statistical measure showing the degree to which one data series leads or lags the other by one period.  This probability measure on the red bars is ½ in the topmost chart (streaks in red bars), and roughly 0 in the chart immediately above (just random noise in red bars).  

Using the same correlation idea on the green and blue lines, of the chart immediately above, we notice a more problematic –½ (the green and blue lines tend to go in opposite directions.)  But it is instead ½ if we look at the correlation made between the change in the blue line and the change in the green line from one period later.  So the forecaster hardly had a “fore”-cast, but rather an “after”-cast.  As useless as an early-warning system waiting until after September 11, to then warn of a possible NYC attack sometime after the next day - September 12.

The finance legend Benjamin Graham joking that those who forecast can only succeed if they shroud their work in witchcraft and mystery, in order to assume value, has documented the issues surrounding these follies many decades ago.  Later, U.S. economist Ezra Solomon made a more explicit comment: “the only value of economic forecasting is to make fortune tellers look good.”

Let’s now discuss some statistical properties of these change errors in the chart immediately above.  We can see the red error bars are less serial-correlative, and more statistically significant across a range of the 25 years.  The coefficient of variation (of variation per unit of average) is steady at a highly significant level.  The distribution of the red bars are also less negatively skewed (asymmetric), which we can summarize in the chart below.  The negative value bias in the skew (or asymmetry) of the errors is far more normal in our change approach.  See the chart below for the distribution of the red bars from the two charts above.  The skew has units of -1.6 on the left distribution, while -0.6 on the right distribution.  The further negative the value, the more caught off-guard leading PhD economists (alongside all other people) are to adverse economic changes.  This is evidence of the stickiness of stable forecasting models that executives still use, as described by Focardi and Jonas.  It is an issue that Taleb terms “Black Swans” in reference to why no one, including business and policy officials, can map the future.

The right-side distribution below still has less of this downward skew and otherwise appears more of a normal distribution.  This implies that any businessperson, who presumes to be able to predict the macroeconomy, instead has foresight worth absolutely, nothing more than an erratic random number generator!

The conclusions from this research here show that despite its importance, it is next to impossible to add significant value from macroeconomic forecasting.  The errors made by trying are just conditionally worse when they matter the most, during large turns in the economy.  Company presidents and their business stakeholders are better off focusing their energy on better areas of economic value maximization.  Such as broad innovation strategies, targeted product research, and the risk management via stress tests.  Or just spare financial lenders the monetary distraction.

What we learn from efficient market theory is that if exceptional forecasting were possible, then enough important people would do it.  Opportunities to take advantage of the economic cycle would be exploited.  The economic cycle would smooth out.  And then leave nothing for anyone to forecast.  Instead we have the same business cycle duration for nearly a century, and economic forecasting is getting worse (overly relying on looking at recent history instead of actually predicting the future).  The Great Recession showed how business leaders were all collectively ignorant in this regard.  There was a hasty retreat from the now seemingly myopic choices some top executives were vainly worried about (such as executive office decorations, details of photo opportunities with celebrities, corporate-paid personal parties, and far-flung projects away from the core business).  To be sure, we see through the Administration’s bailout programs, that many companies’ shareholder value, and as an extension the hard-earned and less-protected pensions of their employees, were severely impaired by not focusing on the areas where one can genuinely add value.  And there is no reason to believe the unwarranted hubris has been shaken out.