Systematic Trading with Rob Carver
Robert Carver is an independent systematic futures trader, writer and research consultant; and is also currently a visiting lecturer at Queen Mary, University of London. He is the author of “Systematic Trading: A unique new method for designing trading and investing systems,” and “Smart Portfolios: A practical guide to building and maintaining intelligent investment portfolios”.
Until 2013 Robert worked for AHL, a large systematic hedge fund, and part of the Man Group. He was responsible for the creation of AHL’s fundamental global macro strategy, and then managed the fund’s multi-billion dollar fixed income portfolio. Prior to that Robert worked as a research manager for CEPR, an economics think tank, and traded exotic derivatives for Barclays investment bank. He spent his early career in the Middle East.
Robert has a Bachelors degree in Economics from the University of Manchester, and a Masters degree, also in Economics, from Birkbeck College, University of London.
Liza: Your book Systematic Trading is mainly about removing the emotion from your trading. What emotional mistakes affect traders most significantly?
Rob: The big things that people talk about are greed and fear. I’m sure you can easily think of quite a few examples– imagine somebody who was lucky enough maybe 18 months ago to buy into the cryptocurrency boom as things were rising. They may have thought, “I want to make millions out of this,” and remortgaged their house and put all of their net worth into it, ignoring the risks involved. That’s not a story that necessarily would have ended well, depending on where they got in of course. And this is a common pattern, the same thing happened in the dot com boom of the late 1990s, and again going into 2006-2007.
And then you have fear which is when people panic, essentially. Going back to the very sharp market correction in February of this year as an example, on a one-day basis that was a reasonably big market move. But if you look dispassionately and look at the statistics and think about the longer term, was that by itself a reason to sell all of your stocks and buy treasuries and gold bars and put them under your bed in sheer panic? Probably not but a lot of people got scared enough that they actually did that, they panicked on what was not an especially large move. So essentially I think greed and fear are the main emotions that blind people.
Liza: Is that mostly what the book is about? Can you tell us more about it?
Rob: It’s essentially laying out the story as to why you should trade systematically. One big part is the fact that human beings can’t actually analyze larger and more complex data than computers can, at least at the moment. Humans have emotions and they can be decent traders but for most of us, including myself, the emotional side overwhelms us and we’re better off with a trading system. Trading systems are nothing new, people are writing books and putting things on websites and trying to sell them to you all the time. But most of my book is about the principles of designing trading systems and avoiding the mistakes that people tend to make, which form three big categories.
The first one being overfitting – that’s where you get the data from the past and you fit your trading system too closely to that, essentially thinking the past repeats itself exactly. Now you can’t really trade systematically unless you believe that the past is some guide to the future, but there is no way it’s going to happen in exactly the same way.
The second mistake traders tend to make it that they make bets that are too big. They put on concentrated positons in a small number of assets with too much leverage. Going back to the previous question, someone who puts 100% or more of their net worth into Bitcoin is doing something pretty crazy because that’s a very concentrated and highly leveraged position. Most people, particularly on the retail side, tend to trade with too much leverage and they shoot for returns that even an institutional trader would be wary of because of the risk involved.
The third category of mistake is what I call overtrading. This is where you’re buying and selling on short time horizons without thinking about whether you actually have an edge on that time horizon or whether the costs, commissions, and the slippage you’re paying overwhelm your returns.
So the book is a deep dive into the moving parts of a trading system and how they fit together and how to build each of them to perform in a way which hopefully, if you’re lucky at least, make some money.
The book is a deep dive into the moving parts of a trading system
Liza: Can you talk a bit about how you adjust positions to handle market risk?
Rob: The first thing you have to think about is the level of conviction you have in a particular trade. A lot of people have what I call a binary trading system where basically they are long this and short that, without thinking about how much conviction is involved.
The second thing you need to think about is how much risk you actually want to take in your portfolio, which goes back to leverage. You also have to think about things like how much leverage the broker will give you, how much margin you may need, and the possible returns you might make.
Then you try to measure the risk of the asset you’re trading. Risk or volatility is probably the thing that people in quant finance know best how to forecast. We’re really lousy at predicting future average returns of assets and we are a bit better at predicting correlations, but we’re pretty good at forecasting risk and that’s because risk tends to be quite stable. If you measure volatility over the past 30 days that’s a pretty good guide to what volatility will be over the next 30 days. You can make fine tuning adjustments to that and then the problem you might encounter would be when volatility of the last 30 days doesn’t reflect what’s going to happen next because there’s an event coming up. If there’s a big event you know about, such as an election or a news announcement like a rate change or nonfarm payroll, then you might want to adjust your risk to account for that increase in volatility.
Liza: You’ve written a Python backtesting system called pysystemtrade. Can you tell us about that?
Rob: In my previous role I’d been working as a quantitative portfolio manager so I had other people (who were much better at programming than me) to actually build software and plug it into a system that does all the other important things, like get price data and execute trades in the markets, none of which I had to worry about. I thought it would be fun to build my own trading system from scratch, so I implemented such a system a few years ago for my own personal capital. A couple of years ago I went back to that project and was appalled at the quality of my code and decided to rewrite it. I thought it would be fun to release it at the same time as an open source project so that other people could laugh at my code and make suggestions for improvement, so that’s been an ongoing project. It’s been slow-going because I’ve been doing other things but it’s probably within a few months of being something you could actually use as a complete trading system.
Liza: You mentioned curvefitting being one of the three major mistakes that traders make. How do you avoid that?
Rob: It’s probably impossible to avoid completely. I consider the way most people curvefit or overfit their trading models into three categories. The one that most people think about is what I would call explicit fitting, which is where the fitting is done explicitly by some automated process. A very simple process could be a grid search, which is essentially taking the range that parameters in a system could take and generating a backtest for each combination and then simply picking the best one.
The second way that people fit is what I will call implicit fitting. What typically will happen is that someone will generate a backtest for the given set of parameters. Then they look at the whole history of backtest and they say, “I reckon I can make it a bit better, I’ll just go back and change it.” That ability to look at the whole history and make that change would not have been available if you’d actually run the trading system live. You’ve basically used a time machine. This is a really dangerous way of backtesting and overfitting because it’s harder to control or know how much you’re doing wrong if you’re wrong. It’s a sin that everyone commits all the time including myself. Every time you backtest a new idea and you look at the performance and if it’s not good and you abandon that idea, you’re essentially doing implicit fitting because the trading system that you will run won’t be the one you would have run if you didn’t have the information.
The third fitting is called tacit fitting and this really is impossible to avoid. The idea here is that everyone that has read or taken a course in college or watched a video on YouTube or been on anyone’s website will have some idea of the kinds of things that historically worked well in trading. And if you then sit down at the computer and test one of those ideas you are using information that wouldn’t have been available in the past to somebody who hadn’t read all those papers or hadn’t watched of those YouTube videos because they hadn’t been made. So again this is the use of a time machine. But apart from lobotomizing yourself, if it’s difficult to learn how to get around this one. Firms sometimes employ people who are from non-financial backgrounds like physics or engineering or statistics because those people tend to be pretty smart but also because they are less likely to have this tacit knowledge. In terms of what I do myself, I know that tacit fitting exists so there’s not much I can do about it, and it’s probably getting worse because every day I read more books and papers that other people have written. I try to avoid implicit fitting if I can try and do things in an explicit way which makes it easier to control.
Grace: How do you go about finding optimal complexity for a strategy?
Rob: As we add more complexity, more degrees of freedom to a strategy, we make it more likely that we’re going to curvefit. Now if we have too little complexity in the strategy, too few degrees of freedom, then maybe it’ll be too simple and won’t do the job we need. It’s a very difficult question to answer because one thing you could do is just run a bunch of experiments with some common strategies with different levels of parameters, complexity, and degrees of freedom and just see how the optimum level of complexity comes out. And I have done this myself but with a fairly simple example which is looking at the momentum measured with moving average crossovers. That’s essentially a two parameter model and if you restrict the model so that you can only pick one parameter, the out of sample performance improves. So as a rule of thumb, the optimal level of complexity is probably less than you can imagine. Most people allow too much complexity and too many degrees of freedom when they are fitting. I actually prefer to do my fitting in a portfolio weighting space. What I mean by that is rather than having one trading rule which might have a dozen parameters and be potentially very complicated and overfitted, what I have instead is a number of trading rules with no parameters. This sounds crazy but what that means is there will be a simple trading rule with maybe one or two parameters and those parameters are prefixed at given values. If you think about that in the momentum space that might mean for example I have perhaps a slow, medium, and fast sort of moving averages to measure my trends speed. And then what I do is work out what the optimal portfolio weights are for that portfolio trading rules.
So as a rule of thumb, the optimal level of complexity is probably less than you can imagine.
Grace: What are the steps that you take to develop a trading strategy?
Rob: I begin with an idea, and this might be an idea that I’ve stolen off somebody else or occasionally it’ll come to me in a dream but normally it’s based on either reading someone’s paper or perhaps seeing a particular movement in the market and thinking, “Well I wonder if this is a common pattern that can be exploited.” So I’ll give you a simple example. When I first built my trading system it was very simple so I was looking for things to add to it and I decided to add a breakout system. Then I had to think about what to use to measure breakouts. There have been voluminous numbers of pages written on this but if you get back to simple principles essentially you need a way of measuring the range of price movements over a given period and flagging when a price moves out of that range. For example, using it like a rolling window, find the maximum and minimum price over the last 250 business days, which gives us the range. Then it’s very easy to come up with a simple trading rule that measures where you are in that range, including whether you’ve broken out of it.
Then I do something slightly unorthodox– I test my ideas using fake data.
Then I do something slightly unorthodox– I test my ideas using fake data. At this stage, I’m not really interested whether the trading rule will perform well with real data because I’m trying to avoid explicit or implicit fitting as much as possible. So I construct fake price series where the trading rule should do well or should do badly, for example a breakout system should do well in times when prices are trending and will probably do badly in times when prices are choppy. I’ll generate some fake data that looks like the real prices and has similar volatility characteristics and has very strong trends or very strong range-bound behavior. I then calibrate my trading rules to work out the optimal parameters based on things like the costs that I’m paying, but also the performance with respect to the kinds of trends I’m trying to pick out. So I might say, “Well I think this breakout rule is going to try and pick up a six month trend,” and create a price series artificially with six month trends and then tune my parameters until I end up with something that does well with a six month trend. And that’s how I get to the point where I have lots of trading rules that are all very simple and have preset parameters. I also look at whether the costs of these rules will be too expensive or too cheap for the kinds of instruments I’m likely to be trading. Then I consider the correlation of the returns of the rules, because if they are highly correlated I probably have too many of rules with similar parameters and I can prune some. I want to end up with perhaps half a dozen of the things that I want to pick up trends with lengths from perhaps a week to a year. Then I put all of these in an optimization which will choose the correct weights for them. This is now an explicit optimization, which can be done in a robust way, out of sample with no use of time machines. It’s only at this point that I’m actually using real data. The idea is to use real data as little as possible and to do as much design work up front using fake data.
Grace: What metrics do you use to judge the quality of a trading strategy?
Rob: Well the obvious one is the Sharpe ratio, however I tend to think about things in a very probabilistic sense. I won’t just look at the actual size of the Sharpe ratio, instead I will look at its statistical significance. To see why this makes sense, think about two trading systems. One makes no money for ten years, then in 2008 it makes a lot of money, then for the next ten years it makes no money. The Sharpe ratio could be quite attractive because measuring the risk of that strategy, most of the time it’s not making or losing any money so the risk is very low and then it’s got this big return in the middle. Compare that with another system that makes money consistently year after year with fairly consistent volatility. The Sharpe ratios of those strategies would be similar even though one makes all of its money in one year and the other makes money steadily.
Now if you look at the statistical significance of those two Sharpe ratios what you’ll find is that the first Sharpe ratio is not statistically significant because it’s coming from a single data point. If you shuffle your returns historically to find the likelihood that return would be reproduced you’ll find that is doesn’t happen very often and most of the time you’re making no money at all.
Whereas the second system is much more robust because it’s making money consistently year after year and the statistical significance of the Sharpe ratio will be much higher. So although the Sharpe ratio is a pretty simple metric, looking at the significance really helps a lot and allows you to distinguish between systems that have high Sharpe ratios that you wouldn’t actually want to trade. I don’t really look at things like drawdowns because the drawdown of a backtest is one data point and thinking probabilistically, that’s not something you can draw a lot of significance from. I have looked at things like omega which is obviously a more sophisticated way of judging whether your return distribution is good or not. Very briefly, the Sharpe ratio just uses the first two moments, the mean and the standard deviation of a distribution while omega uses the third and the fourth moments. But I can still construct account curves that have really good omegas that I wouldn’t want to trade because returns aren’t consistent enough. Generally I stick with looking at the significance of the Sharpe ratios to judge whether something is worth looking at.
Liza: What advice would you leave for quantitative traders that may be new to the industry?
Rob: I am not arrogant enough to think I can say anything new, however I would say the one thing that most of us forget about is that as quant traders we are potentially exposed to being overconfident because a lot of us are competent, highly qualified, have gone to good schools and consider ourselves to be high-I.Q. people. But I think we still are always in danger of overestimating just how good our trading systems and our results are. We forget about the fact that statistical significance is usually much weaker than humans intuitively expect. It’s only when you go back and do the math and look at the results and ask yourself that hard question — I built this trading system and it looks amazing but actually is that result statistically significant? Is it significantly better than the trading system I’ve got? Is it significantly better than the benchmark? You can have backtests that look really good but when you actually look at the statistical significance you may find it is very weak or nonexistent and that doesn’t mean to say that we shouldn’t actually run these trading systems because they may well have a well motivated idea behind them that makes sense. But if that’s not the case and you have just spent a few weeks merrily curvefitting your way through the data, you may have come up with a system that looks amazing but you can’t explain why it works or you can’t explain what it’s doing or the economic drivers behind why it might be profitable. When you actually dig down you find that results are amazing, but because of the overfitting the statistical significance isn’t there. I think everyone in this industry should write on a post-it note three words “Is it significant?” put that at the top of their monitor and just keep looking at that the whole time.
Everyone in this industry should write on a post-it note three words “Is it significant?” put that at the top of their monitor and just keep looking at that.
Liza: Statistical significance seems to be a concept we keep coming back to.
Rob: Most good quants use statistical significance in their systems, all quants know the theory but many forget it or forget about it or get carried away with what they’re doing because it’s all so exciting. Good quants think about this stuff all the time but maybe don’t talk about it so much, perhaps because it seems trivial compared to the more fun and interesting stuff we can talk about. But it’s a very fundamental part of the business that I think it’s the one thing that most people don’t pay enough attention to.
Build a trading system of your own!
Get your free token to connect to the REST API and start trading. To generate your token:
Risk Warning: The FXCM Group does not guarantee accuracy and will not accept liability for any loss or damage which arise directly or indirectly from use of or reliance on information contained within the webinars. The FXCM Group may provide general commentary which is not intended as investment advice and must not be construed as such. FX/CFD trading carries a risk of losses in excess of your deposited funds and may not be suitable for all investors. Please ensure that you fully understand the risks involved.