Best of TTU – Track Records vs Simulations.. What’s Best?

For those of us who have been on the manager side for a while, you'll know that investors love to analyse decades of performance data before deciding which manager they are going to invest with, and that makes a lot of sense.  But we also come across investors who subsequently redeem based on just a year or two after they invested, which is usually just down to bad luck and unfortunate timing.  This does not seem logical, but it does relate to the short conversation I want to share with you today, where I discuss the role and importance of track-records & back-tests with Scot Billington.  We also ended up discussing an interesting twist to their research, which led them to abandon taking any Short Trades in their model.  So enjoy these unique insights from Scot, and if you would like to listen to the full conversation, just go to Top Traders Unplugged Episode 25 & Episode 26.
The Long side vs the Short side
Niels:  Now, track record...we've touched a little bit upon the track record. What I'd love to do is to ask you how one should read your track record because we all know that strategies evolve over time, and, therefore, in a sense, one could say that actually a track record...sure it shows that you survived. It shows that you have had some innovation, but I think sometimes people maybe get fooled to believe that a track record is a great indication of what the future is going to look like because they really don't know what changes have happened along the way in the model. So in some ways one could say that maybe it's better to look at the backtest of the current model when you look at a manager, maybe that would say more about the future, I don't know, but I'd love to hear about what your view and what your observation is about your own track record, because you mentioned the short side of things, and I know there was a period where I think you didn't take any short trades at all, so I'd love to hear your philosophical view about short trades, because I don't think many people realize that there is a big difference between the long side and the short side in terms of success and profitability, but also generally maybe putting that into context about your own track record and why you made the changes along the way?

Scot:  Well I think you make an excellent point about track records. I think people look at them as some kind of a one loss record in a sporting event, and they miss the enormous amounts of variance and randomness that happened to have lined up and occurred to produce whatever monthly return is shown, or daily return. I think that most groups don't do enough qualitative analysis of the trading method, and simply crunch performance numbers as though they're the end all be all of what the future is going to be. You make an interesting point. I would probably argue that a backtest of the current running model is probably the best, except that it is also going to...if you are hoping to be a 5 or 10 year investor, there're going to be future changes. And so what I might say is, as we've progressed, our changes have been successful, and therefore perhaps one would conclude that our future changes would also be successful. That's a different discussion. Our first major change to the model was in early 2002, was basically eliminating short trades and using a volatility filter for long trades.

'Niels: So why did you make a change to eliminate Short trades back in 2002? Scot: Well, a Short trade is bounded by zero, so it's going to face several hurdles'

Niels:  Were the short trades the cause of your drawdown back in the beginning?
Scot:  No, no.
Niels:  So why did you make a change to eliminate short trades back in 2002?
Scot:  Well, short trades, and this is particularly appropriate in our timeframe, on a shorter timeframe I don't think this would not hold as much. Let's imagine in our timeframe we're trying to hold a good winner fo...