Extreme Value Theory and Backtest Overfitting in Finance

This paper seeks to develop minimum standards for backtesting in order to reduce curve-fitting and ensure strategies are picking up on market signals rather than market noise. Oftentimes, strategies will do well simply because they interpreted random market noise as a signal, so while this strategy may backtest well it will not perform in a live market environment. This paper uses extreme value theory to determine the probability of observing a specific result of multiple configurations of a backtested strategy.

Several indicators could be used to measure the risk and volatility a strategy takes on, including the Sharpe ratio, which is used in this paper. In this study, a backtest is considered over fit if a non-zero Sharpe ratio is produced in-sample while the strategy has an out-of-sample Sharpe ratio less than or equal to zero. An example of an overfit strategy is demonstrated using a tool that shows how simple it can be to optimize a strategy to fit a randomly-generated set of numbers. An extreme value distribution is a type of function that has specific convergence properties, so it is used to analyze the maximum Sharpe ratios for a collection of trading strategies.

A Sharpe ratio rejection threshold designed to prevent false discoveries is established and tested using probability distributions. However by nature it also increases the chance of a missed discovery due to the slow convergence of normally distributed random variables in the function used. Based on previous work this paper uses results from extreme value theory to develop a minimum backtest length in years of data that is necessary to increase an investor’s confidence that the results of the backtest are not due to overfitting. This value, (MinBTL), is the number of years of market data needed for an optimal backtest, and is derived using several distributions including a Gumbel distribution.


Risk Warning: The FXCM Group does not guarantee accuracy and will not accept liability for any loss or damage which arise directly or indirectly from use of or reliance on information contained within the webinars. The FXCM Group may provide general commentary which is not intended as investment advice and must not be construed as such. FX/CFD trading carries a risk of losses in excess of your deposited funds and may not be suitable for all investors. Please ensure that you fully understand the risks involved.