This note looks closely at the probability models associated with being a successful performer, defined as consecutively outperforming the typical benchmark performer. S&P Dow Jones recently tabulated a research table, of broad market fund returns, showing the portion that continuously perform in just the top-half, year after year.
Before showing those results, let's review some probability theory concepts. Take a small hypothetical example where we have a sample of 4 funds, and we then track those managers over each of two years.
Example 1. If there is positive persistence, then 50% of the original funds (or 2) would be top-half performers over both years. These funds are colored green below. This is the most likely example for having more eventual fund failures of those in the bottom-half.*
fund 1 --> top-half in year 1 --> top-in after year 2
fund 2 --> top-half in year 1 --> top-in after year 2
fund 3 --> bottom-half in year 1 --> bottom-half in year 2
fund 4 --> bottom-half in year 1 --> bottom-half in year 2*
Example 2. If there is no persistence (a random haphazard performer), then 25% of the original funds (or 1) would be top-half performers over both years. Or 50%^2. These funds are colored green below.
None of the probability theory here would say then -both indisputably and critically- that the cause of the selected managers' success is due to luck as opposed to skill, since we have shown using Bayesian statistics in the links above (and Market's downward tilt) that at some level here we have made the case that such managers have about as much skill and luck level in the resulting 5-year performance (even more level of skill could be expected for top-quartile and even more likely that small group was skilled). And even skilled managers do not continuously score flawlessly, above average, but we can expect then to "only" average such outperformance over more considerable time.
Before showing those results, let's review some probability theory concepts. Take a small hypothetical example where we have a sample of 4 funds, and we then track those managers over each of two years.
Example 1. If there is positive persistence, then 50% of the original funds (or 2) would be top-half performers over both years. These funds are colored green below. This is the most likely example for having more eventual fund failures of those in the bottom-half.*
fund 1 --> top-half in year 1 --> top-in after year 2
fund 2 --> top-half in year 1 --> top-in after year 2
fund 3 --> bottom-half in year 1 --> bottom-half in year 2
fund 4 --> bottom-half in year 1 --> bottom-half in year 2*
Example 2. If there is no persistence (a random haphazard performer), then 25% of the original funds (or 1) would be top-half performers over both years. Or 50%^2. These funds are colored green below.
fund 1 --> top-half in year 1 --> top-half in after year 2
fund 2 --> top-half in year 1 --> bottom-half in after year 2
fund 3 --> bottom-half in year 1 --> top-half in year 2
fund 4 --> bottom-half in year 1 --> bottom-half in year 2
Example 3. If there is negative persistence, then 0% of the original funds (or 0) would be top-half performers after 2 years.
Example 3. If there is negative persistence, then 0% of the original funds (or 0) would be top-half performers after 2 years.
fund 1 --> top-half in year 1 --> bottom-half in year 2
fund 2 --> top-half in year 1 --> bottom-half in year 2
fund 3 --> bottom-half in year 1 --> top-half in year 2
fund 4 --> bottom-half in year 1 --> top-half in year 2
The failures in Example 1 are shown with the one-way, absorption state in the lowest row of the probability matrix below. This is a fourth quartile performer throughout the chained process, while the other quartiles share their 25% portions starting with the top quartile.
So the universe in the S&P Dow Jones report was just over 2860 funds, and covers a time frame of just the recent 5 years. A random distribution of haphazard fund managers, similar to Example 2 above, would reveal 3% of them as being top-half performers in each of 5 years. Or 50%^5.
The failures in Example 1 are shown with the one-way, absorption state in the lowest row of the probability matrix below. This is a fourth quartile performer throughout the chained process, while the other quartiles share their 25% portions starting with the top quartile.
So the universe in the S&P Dow Jones report was just over 2860 funds, and covers a time frame of just the recent 5 years. A random distribution of haphazard fund managers, similar to Example 2 above, would reveal 3% of them as being top-half performers in each of 5 years. Or 50%^5.
But the research results on Exhibit 2 of page 3 of that report show it is instead 2% (4.47% of the first 50% cut). Or something closer to Example 3 above, than to Example 1 above. Sure 2% and 3% are both very small numbers. And a New York Times article, which had introduced the underlying research table, speaks well to the empirical smallness of these numbers and the funds behind them. But an interesting question, more apt for those interested in probability theory, is how do we make sense of this difference? After-all 3% is small enough through random chance alone; now we're down to 2%.
Another interesting characteristic of the research table, is that S&P Dow Jones disaggregates the funds into mutually exclusive and completely exhaustive sub-categories. Using size benchmarks of Large, Mid, Small, and Multi. So see the performance results in either chart below (the second chart below is simply the same information as the first chart but with a log-transform on the vertical y-axis).
Notice that for these empirical colored lines, the survival portions from all four, fund sub-categories are all below that from the haphazard fund manager (the grey dashed line)? Not one category showed persistence performance equal to what a group of haphazard fund managers would, let alone positive performance above that level that a skilled managers would want. Further we see how parallel these fund categories are, below the haphazard's dashed line.
This isn't the most revolutionary science, but even still there is a risk that without looking at specific probability characteristics, one can make inference mistakes. One of the big areas for one to consider in this case is whether we have a large sample size to begin with. This way we are safe from making too much of any movements from small numbers.
But for this research study, the entire universe was nearly 3000 funds, which is certainly large. It is categorized as the following: about 1036 Large, 444 Mid, 620 Small, and 761 Multi. Now for most of these years, one can keep taking the top half or so of funds and still have a respectable sample size remaining. But we won't have a significant sample size forever, of course. Ultimately the final number of funds that retained top-half status for all 5 years is ok. About 64 for the empirical study (2.2%*2862), and 89 for a haphazard manager (3.1%*2862). For more important ideas on how these probabilities could work over an investment horizon, see the additional notes on the bottom of the most popular The forever elusive α, which also covers some similar and compelling ideas (on an aside our second most poplar web log article The incapable soothsayers, was written just several weeks ago.)
A similar analysis on these funds in the S&P Dow Jones report, but highlighting top-quartile funds, is not shown here given how small the ultimate universe is (not the number of years since that's ok) and the weak confidence interval about showing those extreme results over time. And this also bumps up against another take-away, for our probability article here. That is regardless of the approximate cause for these 64 fund managers' success, during the past 5 years, nearly 97% of them won't be able to continue this streak over the subsequent 5 years. An important aside, for funds that existed for more than the recent 5 years, evaluation of those funds should incorporate all investment history for that lengthier historical time.
We have just seen that every 5 years, we only have about 3% of any group able to claim success using as a threshold this consecutive, top-half performance. And 3% of 64 funds would imply about 2 funds (out of an original list of nearly 3000). So without going too much further in time (and only 10-years total), we see the same small number of dwindled funds that are able to succeed, as are top-quartile for each of only a 5-year period:
50%^10 = (50%^2)^5 = 25%^5
50%^10 = (50%^2)^5 = 25%^5
None of the probability theory here would say then -both indisputably and critically- that the cause of the selected managers' success is due to luck as opposed to skill, since we have shown using Bayesian statistics in the links above (and Market's downward tilt) that at some level here we have made the case that such managers have about as much skill and luck level in the resulting 5-year performance (even more level of skill could be expected for top-quartile and even more likely that small group was skilled). And even skilled managers do not continuously score flawlessly, above average, but we can expect then to "only" average such outperformance over more considerable time.
No comments:
Post a Comment