Does Back Testing Help You Make Money?

Everyone thinks they can find a solid rule based way to make money in the stock market. It is just a question of analyzing all the data from the last 100 years or so and discovering what works.  Given the availability of the data and the computer power available, it should be easy enough.

The idea has some flaws however.

The idea that we can predict the future based on the past is silly for two reasons.

  1. The past never encapsulates all of the information that made it happen the way it did. Most of the past is a summary.
  2. The future can and will include important things not present in any of the data from history.

Rule based systems are not as easy to implement as people think. Even if you were to find one, you would have to follow it exactly. I have often said that if I could devise a sales system that resulted in a close on recommendation 10 out of 10 times, it would never work 20 out of 20. People would tinker. Highly predictable is boring. Discipline is a scarce resource.  What happens if it suddenly doesn’t work?

The stock market is not deterministic. It is too easy to assume that things will happen as you expect.  They don’t. At best, you have an approximation of what has happened. Your system may say something like switch from small banks to ice cream vendors, but maybe there is no buyer for your stock or maybe not enough is offered at the price your system assumes. Trading costs and taxes matter. Chaotic systems like the stock market all have one characteristic. If anything changes, they are not the same system anymore. The problem was best stated by Edward Lorenz.

“Chaos: When the present determines the future, but the approximate present does not approximately determine the future.”

The worst result of a back test that seems to work is overconfidence. There is an old idea in Las Vegas that says, “We will send a jet to Australia for someone with money and a betting system they believe in.” Why? Because such systems fail eventually.

Ben Carlson wrote a piece recently about things you cannot learn from a back test. You can find it here. Pay careful attention to how people cannot understand what it feels like by looking at a spreadsheet. Every portfolio will have a book loss in the future.  How it feels matters.

Ben adds a useful piece of information.  “It is easier to back test the past than the future.”

Don Shaughnessy arranges life insurance for people who understand the value of a life insured estate. He can be reached at The Protectors Group, a large insurance, employee benefits, and investment agency in Peterborough, Ontario.  In previous careers, he has been a partner in a large international public accounting firm, CEO of a software start-up, a partner in an energy management system importer, and briefly in the restaurant business.

Please be in touch if I can help you.  866-285-7772

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )


Connecting to %s

This site uses Akismet to reduce spam. Learn how your comment data is processed.

%d bloggers like this: