Why Past Performance is not Indicative of Future Returns

Professor Matthew Spiegel

February 5, 2006

Everyone has seen the famous government mandated disclaimer under nearly every investment performance chart, “Performance data quoted represents past performance and does not guarantee future results.”  Actually, for most funds, newsletters, and advisors a more accurate quote might be, “Performance data quoted represents sheer luck and is not indicative of anything else.”  Skeptical?  Well read on!

Returns May be Higher in the Rear View Mirror than they Actually Are

You are busy looking for mutual funds that you believe will do better than the market as a whole.  Where to start?  What if you begin by looking at the recent and long term history of all funds?  The table below lists the average return to all no load funds that had as of October 2005 at least 80% of their holdings in U.S. stocks and compares it to the overall market.

Wow!  Finding funds that do really well is going to be easy!  While there appears to be some fluctuation in how funds do relative to the market over time, they are clearly ahead on average.  It sure is nice to have all of these really talented stock pickers to choose from!

One second.  The returns listed above are for funds that were alive as of October 2004.  But that is what we want right?  Funds we can invest in.  Not quite.  Funds with poor records often go out of business.  The problem is that as of today you do not know if your fund will dissolve in the next few years.  This means that if you only look at funds that are alive today to calculate past returns you are using information nobody in the past had.  Statistically, the live funds as of a particular date form a biased sample; one with returns that are higher than you can expect to earn.  The formal term for this is “survivorship bias.”

A simple example, from an old scam, illustrates just how serious survivorship bias can be.  Despite your best efforts you discover that you have absolutely no stock picking talent whatsoever.  Unfortunately, you have as your life’s goal the management of other people’s money.  Fortunately (?), you are also a tad unethical.  What to do?  In January 2003 you open up your office and pull out the local phone book and select 320 names.  You divide them into two halves.  To each group of 160 you send the following letter, “Let me introduce myself.  After years of studying the stock market I have opened an investment management office in your town.  Since you do not know anything about me or my talents I am offering you (FREE OF CHARGE) my number one stock selection this month:  I recommend you buy Nortel Networks.”  The other half of the phone book receives the same letter but a recommendation to sell Nortel Networks.  What happened?  During the month Nortel went up 30%. Good news for those you told to buy, but not such good news for those you told to sell.  Now what?

At the end of January half the people (all 160 of them) you sent letters recommending to buy Nortel Networks think you might be a pretty talented investment manager.  After all your “advice” was right, the stock you told them to buy went up.  Next you take half of the winning group and send them a new letter, “Last month I suggested you buy Nortel and it went up 30%!  Since you may still wonder about my talents I am sending you yet again (FREE OF CHARGE) my stock pick for the month:  Buy JDS Uniphase.”  Of course, the other 80 people get a letter saying to sell.  This time the results are less spectacular in that the stock only rose 2% over the month.  Still, not bad for a single month.  In any event, you now have 80 potential clients for whom you have given seemingly prescient forecasts over the last two months.  So you send out one last free letter recommending to 40 of them that they buy Tyco and to 40 that they sell Tyco.  This time the stock goes down by 10%.   Now, the letter that you hope will close the deal goes out to the 40 families to which you have sent three “accurate” calls, “Like anybody else I need to make a living and can no longer offer my services for free.  Over the last three months the stock recommendations I sent to you have earned a compounded return of 47%!  If you would like to continue receiving my valuable advice please contact me about setting up an account.”  Of course, this little scheme is illegal.   Well, at least it is illegal if you do it as transparently as suggested above.  It is, however, perfectly legal if you do almost the equivalent but via a somewhat less obvious path; more on this in a bit.

What do our nefarious investment advisor’s actions have to do with survivorship bias?  Why everything!  The advisor’s scheme works only because of survivorship bias.  Each family that receives a letter sees only their letter, and they only continue to receive letters if the previous letter’s recommendation turned out to be accurate.  Thus, the 40 families that received a letter claiming that they would have earned 47% with the recommended picks are the survivors from the original 320 families that received letters.  If instead each family saw all of the letters that went out they would not be fooled.  Instead they would see just how worthless the letters really are.

For better or worse, the returns produced by the mutual fund industry have a lot in common with those generated by our disreputable investment advisor.  Mutual funds and other institutional investors comprise a very large fraction of the overall market.  Thus, it is not uncommon for one fund to sell stocks to another.  At the end of the day the results are identical to that of the worthless newsletter.  The investors in the fund on the winning side of the trade will see market beating returns and those on the losing side below market returns.  In the long run, “winning” funds will see inflows and “losing” funds outflows that will eventually cause them to shut down.  If you only look at the survivors, then you will only see the funds on the winning side of each trade.  But investors three years ago did not know which funds would be on the losing end and shut down by today.  Had they known, they would have stayed out of the losing funds!

When analyzing a strategy for selecting stocks, mutual funds, or any other investment you should look at how it would have performed in real time.  That means you cannot test it simply on the securities that exist today but must instead test any investment strategy on the securities that existed at the time it might have been used.  If you want to test a mutual fund strategy that means you need to go back and try it out on the funds that existed in each period you might have wanted to use it in the past.  Supposedly then, good strategies will steer you away from the funds that eventually go under.  But you will not know that unless you include funds that are both currently alive and dead.

Survivorship Bias Free Returns

The first table looked at the returns you would have obtained over various time periods had you invested in the funds that existed as of October 2005.  But there is no known strategy that would have allowed you to do that in any past year.  A somewhat equivalent strategy might have been to divide your money equally among all no load U.S. equity funds that existed at any one point in time.  To be safe, our analysis also needs to exclude funds that have less than a three year track record.  Otherwise you end up with another form of survivorship bias.  Fund companies start numerous “incubator funds” each year with their own money.  They then see what kind of track record they develop over the next three.  Winners are then marketed to the public and losers closed down.  Requiring a fund to have a three year track before including it in the analysis prevents this type of gamesmanship from influencing the results.  Remember we only want to include funds that normal individuals can actually buy.  What happens if you now invest in all of these funds, rather than just those that made it to October 2005?

While the average return over the past five years has been extremely good relative to the overall market’s, it lags over the last one, ten and fifteen year periods.  What makes the long term results even more daunting is that they are probably the more reliable!  Unless a particular strategy generates simply spectacular returns it is difficult to distinguish its returns from luck over any relatively short period of time.  Thus the right conclusion to be drawn from the table is that the average mutual fund has historically produced lower returns than a passive index fund. 

A typical response to the above comparison might be that you are too smart to invest in the “average fund.”  Instead you might seek out “superior” funds that have historically had above market returns over the past three years.  What if you try that?  One potentially helpful source is Morningstar which rates funds on a scale of one through five stars.  If you want to hold the “better” funds you can restrict yourself to those with three, four and five star ratings.  Fortunately, we do not have to test out this strategy ourselves as Christopher Blake and Matthew Morey already did it for us in their 2000 article “Morningstar Ratings and Mutual Fund Performance” which came out in the Journal of Financial and Quantitative Analysis.  The results?  If you just want to avoid really bad funds then the ratings are of some use.  Basically, one and two star funds are just bad funds to invest in.  Their low past returns seem to forecast low future returns.  But at the other end, there appears to be no difference between the three, four and five star funds.  All perform, at best, as well as the market and often worse.  Overall, investing in broad market wide index funds will produce a portfolio with higher expected returns and less volatility than one composed of a few actively managed four or five star funds.

What to do?

It seems clear that comparing one fund’s historical one, three, or five year returns is unlikely to produce a portfolio with above market returns.  One solution is to just put your money into a broad based index fund.  The other, of course, is to consider using the fund selections here at Alpha Investment Opportunities!  Our procedure has been tested on survivor ship bias free data, and in that historical data has generated above market returns.  You can also see how our fund picks have done in real time by clicking on the Historical Returns link in the navigation bar on the left.  As of the date I am writing this our January 2004 portfolio as returned 18.26% versus 8.90% for the market, and the February 2004 portfolio 13.52% versus 8.54% for the market.  Of course, performance data quoted represents past performance and does not guarantee future results.

© Matthew Spiegel