Understanding How Fund Performance Comparisons Overstate Returns

May 13, 2014

Every investor has, I hope, read the standard disclaimer on mutual fund and ETF performance documents that ‘past performance does not predict future performance,’ or other text to that effect. Still, when you read a fund company’s statement that ‘x% of our funds have out-performed their category average’ or that ‘x% of our funds are rated 4- and 5-star by Morningstar,’ this seems meaningful. Similarly, if you read that the average small value fund has returns ‘y’% per year over the last 10 years, this also seems significant. There is a major factor that is missing in many of these types of comparisons of fund performance, however, that tend to make active funds look better than they really are (as compared to low-cost index funds) as well as making a mutual fund family’s managers look more skillful than they might actually be.

The often-overlooked factor in these types of comparisons is what is called survivorship bias. The mechanism is quite simple. Imagine that a fund family has ten domestic stock mutual funds that it manages. The funds are managed using different strategies. Half of these have out-performed their category averages over the past five years, but the other half have under-performed their category averages. The amount of assets in the poor performers has fallen and that in the good performers has risen. If the fund family simply closes the poor performers, so that only the half that are out-performing remain in existence, the fund can quite rightfully claim that 100% of its funds have out-performed the category averages over the past five years. The closed funds simply no longer exist and their poor track records die with them.  If we look at the average return of the currently-open funds, the fund family’s average returns look great. If we know about the other half of the under-performing funds, now closed, and include those in our accounting, the fund family’s performance looks average at best. Ignoring the funds that have been closed and accounting for those that remain open introduces survivorship bias, in which we are only tallying performance of the funds that have survived.

Morningstar reports that its own ‘category average’ returns have a meaningful impact on trailing reported returns. As few as 40% of all funds in some categories have survived the last ten years, by Morningstar’s calculations, and ignoring survivorship bias results in average category fund returns as high as 0.8% per year higher than if the closed funds are accounted for. A more insidious effect of ignoring survivorship bias is that actively managed funds look better than they should as compared to index funds because actively-managed funds are more likely to be closed down. Morningstar finds that the 10-year survivor percentage for actively-managed large growth funds is 42% vs. 74% for index funds in this category. This, in turn, has a major impact on the relative ranking of funds. The same article finds that the Vanguard Total Stock Market Fund (VTSMX), a low-cost stock index fund, is ranked in the top 19% of all large blend funds in existence over the past twenty years, but that VTSMX is in the top 9% of all large blends funds once an adjustment for survivorship is used.

Winston Churchill famously stated that ‘history is written by the victors.’  In the same vein, The Wall Street Journal’s Karen Damato notes that we might say that fund performance tables are written by the winners, those funds that survive. This does not mean that the fund companies are doing anything wrong. A mutual fund company may quite rationally choose to close funds that are not providing attractive returns. On the other hand, investors should know that the fund performance that they see for fund families or for category averages are missing the results for all of the funds that performed poorly and have subsequently been shuttered.

Related Links: