The degree to which the commercial implementation of factors has led to a serious divergence from the evidence produced by academic studies is the subject of some interesting research from Scientific Beta.
The opening premise of the new study is that the market has become increasingly saturated with more and more factors that are, they claim, ever farther removed from the academic foundations of factors.
The perpetual search for commercial advantage is to blame, engendering a factor finding process that has, to date, expanded the number of factors from a suitably academically sparse five to literally thousands of commercially-driven factors.
The authors Felix Goltz, a research director at Scientific Beta, and Ben Luyten, a quantitative research analyst, make the point that in the academic literature stretching back to 1993 and Fama and French's first study of factors, there are three "obvious insights."
One if that different models use identical factor definitions; another is that the number of factors is limited to about a handful of factors; and the final one is that factors are defined by a single variable.
In contrast, the authors point out that MSCI's 'factor box' draws on 41 variables; S&P's 'Factor Library' has 500 variables "encompassing millions of backtests"; and BlackRock markets its Aladdin Risk tool as having thousands of factors.
The authors have an answer for why the academics have eschewed proliferation. First, it has been rejected on empirical terms, and here the authors cite evidence from Hanna and Ready in 2005 which showed that that using 71 factors does not add value over a model with two simple factors (book- to-market and momentum).
Second, they suggest that academic research limits the number and complexity of factors because a parsimonious description of the return patterns is likely to be more robust. "Increasing the number of variables will obviously improve fitting the model to a given data set but will also reduce the robustness when applying model results beyond the dataset of the initial tests," they write.
Spurious factors
It is selection bias that leads to the discovery of spurious factors. That is, "strong and statistically-significant factor premia may be a result of many researchers searching through the same dataset to find publishable result."
This is what is called factor fishing. A key requirement for investors looking at utilising factors as a strategy is the reassurance that the factor identified represents a systematic risk that requires a reward, and that it will continue to do so in the future.
However, Goltz and Luyten say that recent research has shown that it is easy to find great new factors in backtests but such factors add no real value to standard factors. "It is easy to discover new factors in the data if enough fishing is done, but such factors are neither economically meaningful nor statistically robust," they write.
The providers then exacerbate this problem by creating complex composite factor definitions drawing on combinations of variables. This provides for what is called overfitting bias. "When combining variables to improve back-tested factor performance, providers can yet again increase flexibility for capturing spurious patterns in the data," they write.
"Additional flexibility is easily achieved by attributing arbitrary weightings to the variables used in a composite definition," they add. "For a given combination of variables, changing the weight each variable receives in the factor definition may have a dramatic impact on factor returns."
What's the matter here?
Of course many of the providers do claim academic rigour. The authors point out, for example, that MSCI, recently issued a report that states that factor research is "firmly grounded in academic theory and empirical practice" while FTSE also mentions the broad academic consensus that exists for the factors used in their global factor index series.
Yet, provider definitions of factors somewhat give the game away, being more complex than the academic definitions and differing substantially despite using the same label. As the author's write: "A relevant question for investors is whether the 'upgraded' definitions of standard factors, like "enhanced value" and "fresh momentum" add value only in the backtest or whether the benefits hold post publication, i.e. in a live environment."
Further, as discussed many providers then use composite scores in their factor definitions and provide even more flexibility to their factor definitions by making decisions on how to weigh the different variables within the composite.
"Overall, product providers explicitly acknowledge that the guiding principle behind factor definitions is to analyse a large number of possible combinations in short data sets and then retain the factors that deliver the highest backtest performance," Goltz and Luyten write. "In fact, providers' product descriptions often read like a classical description of a data-snooping exercise, which is expected to lead to spurious results."
The danger sin all this are that investors will end up with unintended exposure sin their portfolios. Crunching the numbers, Goltz and Luyten suggest, as an example, that investors who tilt towards a composite quality factor "will end up with a strategy where, depending on the index we consider, only about one third or half of the excess returns are driven by exposure to the two well-documented quality factors (profitability and investment)."
It means that even if the quality factors perform as expected, the performance will not necessarily be reflected in portfolio returns "which are in a large part driven by other factors and idiosyncratic risks."
"Available factor products thus do not deliver on the promise of factor investing," they conclude. "Understanding the factor drivers of returns increases transparency and allows investors to formulate more explicit investment choices. However, being aware of exposures to useless factors, which have no reliable link with long-term returns, is equally useless."
They end by suggesting that a good idea is in danger of being distorted. "For a meaningful contribution to the ability of investors to make explicit investment choices, factor investing should focus on persistent and externally-validated factors. It is time to recall the good idea of factor investing."
