Quant finance is not immune to the occasional fad, and “best strategies” might be one of those. In an uncertain market, one strategy might not work the same way for all trading cases. Trader and author Michael Harris explores the randomness of “best strategies” in this article.
Selecting “best strategies” can result in losses for investors and even some sophisticated traders. Here is why.
Some websites advertise annual returns of “best strategies”. These returns are usually impressive and above buy a hold performance of standard benchmarks. But anyone with basic understanding of the subject knows that “best strategies” is often just marketing based on selection bias. There are two variations of this marketing, the weak and the strong.
Weak version: The best strategies are selected from a large set of fixed strategies to report annual returns. As the number of strategies increases, the probability that a few of them will outperform the market also increases.
Strong version: The set of strategies changes often depending on performance. New optimised strategies are introduced while those that underperform or fail are removed from the set.
In both versions above, only when the number of rejected strategies is reported along with performance, it may be possible to know what is going on. But this is usually never done.
Identifying “best strategies” is possible due to proliferation of tools for the development, optimisation and testing of strategies. Some of these tools are simpler and offer many options for assets and allocations while others are much more sophisticated and include neural networks, genetic programming, and other machine learning capabilities.
Note that there is nothing wrong with using such tools and actually, nowadays it is inevitable for discovering edges. What is wrong is taking the top strategies and reporting their returns. This is because there is high probability, often near 1, that these strategies are random and a result of over-fitting to past historical data. Click here for an article that offers an introduction to the issues of multiple comparisons.
Below is a simulation of 100 random strategies that traded long-only SPY last year based on the outcome of a fair coin toss: If heads the strategy went long and if tails the strategy exited the position. The simulation includes commission of $0.01 per share. Buy and hold return for SPY was -5.25% based on the close of the first trading day of the year.
It may be seen from the above simulation that the mean return of all 100 strategies is -2.53%. However, three of them show returns greater than 17% and the top returned 20.6%.
Based on the above simulation, the top three random strategies outperformed SPY by a wide margin. Is this useful information about choosing a strategy? Obviously, it is not.
The problem for investors and traders
Fixed set of strategies
Choosing in advance which strategies will outperform from a fixed set of strategies is hard. The only sensible approach is either choosing all strategies with equal allocation or a sufficient random sample of them. Both of these approaches have certain complications that are beyond the scope of this brief article.
Dynamic set of strategies
In this case a priori selection may be impossible and any posteriori reporting of top performance is indistinguishable from deception.
What investors and traders should pay attention to among other things
- A large number of available strategies, usually more than 15, is often an indication of potential deception, although not always the case. This is especially true if strategies are uncorrelated, in which case the selection bias increases when reporting performance.
- If there are no reports or announcements of termination of strategies and top performance always includes different strategies.
- Average performance of all available strategies is not reported.
In the case of investors, it is best to contact a registered financial adviser before investing any money in strategies offered online, even if the logic of those strategies is available. Performance always depends on timing of investment and the metrics reported are usually not indicative of what any specific account may realise. Then, there is a possibility of faulty backtest results or in extreme cases, misleading.
Trading and investment contests could be deceiving
Some people still advertise trading records from the 90s. There is no way of knowing if these were due to skill or luck during a period of explosive rally in equities. Given a large number of random unskilled traders, some of them may exhibit exceptional performance by chance alone. Below are results from the same simulation as above but with 10,000 random long-only traders using a fair coin for a strategy.
At least one of them managed to realise 28% return in a down year for SPY. Obviously, that was not due to skill…. But next thing you know is the random trader is trying to sell a track record to investors.
In fact, given a large number of traders, some may exhibit outstanding performance for many years in a row. Click here for an article with an example.
