Year-by-Year S&P 500 Performance

Note: Please read the disclaimer. The author is not providing professional investing advice.

If you desire to develop your own magic trading formula through backtesting or data-mining, you’re going to want to use historical data to test your algorithm’s performance versus at least one benchmark, like the S&P 500.

But annual S&P returns, in my opinion, are not very easy to find. I went to the Standard & Poor site and clicked through a few menus, but all I could find were trailing gains (year-to-date, last year, last 3 years, last 5 years, etc.). Couldn’t get the year-by-year breakdown I was after.

OK, no problem, I thought. I’ll just use Yahoo! Finance’s historical data interface to look up the S&P 500 index (ticker ^GSPC) to see the opening and closing price for a given year.

And since the Magic Formula book has the year-by-year breakdown through 2004, I’ll compute 2004 just to make sure my numbers match Joel Greenblatt’s:

On the first trading day of 2004, ^GSPC opened at 1111.92.
On the last trading day of 2004, ^GSPC closed at 1211.92.
So the 2004 gain is : (1211.92 / 1111.92 – 1) x 100% = +9.0%

And lemme just cross-reference with the Little Book That Beats the Market

Hold on! He lists the Y2.4K return for the S&P 500 as +10.9%, not +9.0%!

But hey, with no formal business training up my sleeve, (albeit one Engineering Economics class), I figured it probably had something to do with dividends not being factored in.

The good news – I was right! The bad news – while Yahoo does track historical dividends for other funds and stocks, they don’t seem to do it for the S&P 500 index. So now I know what the problem is, but can’t fix it. Sheesh!

After some googling I did eventually find what I was looking for, and it was on the S&P site afterall! I still have no idea how to end up there starting from their top page, but who cares, here’s the link:

After opening, click on the “ANNUAL TOTAL RETURNS” tab at the bottom of the worksheet. Otherwise, you’re looking at monthly data.
http://www2.standardandpoors.com/spf/xls/index/MONTHLY.xls

Take a gander at Column E. They match Greenblatt’s numbers exactly. 🙂

And we now have two more data points since 2004:
In 2005 the S&P 500 returned +4.9%
In 2006 the S&P 500 returned +15.8%

snp_performance

If you do the geometric average over 1988 – 2006, you’ll see that the S&P 500 has averaged +12.2%. Not bad since that doubles your money every 6 years!

But speaking of 6 years, if you pick any of the earlier consecutive 6 years, the S&P returns at least 12%, often it’s significantly higher than that. However, if the 6 consecutive years you pick are more recent, they’re not so great – we’re talking low single digits.

Does that mean we’re overdue for a large return? Or we’ve entered a new economic paradigm? Do you seriously think I might know?!

Obviously I don’t, but I can offer some words of advice for those developing a trading algorithm. Step 1, which you no doubt know, is to only even consider implementing your homebrewed strategy (versus using an index fund) if it more often than not outperforms the S&P 500 – and does so with similar or lower risk (standard deviation).

And Step 2… Well since you’ve gone through the trouble to add the S&P benchmarks as a sort of acceptable lower bound, why not also use the Magic Formula results as your target upper bound to beat? The Magic Formula strategy is quick, easy, logical, and (historically) proven. Does it make sense to go with any strategy that doesn’t at least historically beat it?!

If you’ve tried to check some of Greenblatt’s computations yourself, one other (apparent) discrepancy you might see is regarding the average rate of return. If you compute it yourself and get a different number than he did, it’s because you’re using arithmetic average, not geometric.