Investors disillusioned with the performance of active funds and moving into smart beta may end up faced by similar quandaries over the movement of goalposts.
Such is the conclusion of a recent report from analysts at Scientific Beta which suggests the frequent chopping and changing of index methodologies and lack of transparency exacerbates the risk to investors of relaying on strategies which have spurious records.
The authors of the report, Noël Amenc, chief executive at Scientific Beta, research director Felix Goltz and quant analysts Mikheil Esakia and Marcel Sibbe, point out that one of the most common moves by index providers is to “enhance’ their factor definitions.
“When assessing the live performance of an index, it will always be possible to identify a better- performing factor definition for the relevant time period,” the analysts write.
“If providers replace factor definitions because one of many backtested ‘enhancement’ options outperforms the live index performance, investors will not reap any benefits as the improvement will be spurious. Moreover, if providers follow such a data-mining approach, backtests of newly launched indices would indicate performance that is artificially inflated.”
Research Affiliates: Alice’s Adventures in Factorland
More fundamentally, frequent changes to factor definitions call into question the very idea that the factors identified are anything like the persistent driver of returns. The analysts point out that tried and tested factors such as value and momentum, as strictly defined in various academic papers, can be shown to deliver a premium for the identified risk. However, as would seem obvious “actors that require frequent updating cannot be persistent factors and thus frequent updating is a sign of a lack of robustness.”
Further, even if the factor definition and factor selection remain fixed, multi-factor indices may rely on different portfolio construction models to combine factor exposures into an index. Such portfolio construction could, hence, be subject to arbitrary constraints, and the more constraints, the larger the risk to the model.
In broad terms, the Scientific Beta analysts suggest that short-term and ad-hoc adjustments to factor investment methodologies “are at odds” with the aims of long-term investing.
“Appropriate index strategies should reflect this focus on consistent principles and not change methodologies according to the latest fad,” the team write. “Moreover, there is evidence that when investors change their exposures frequently over time, these efforts typically have adverse results.”
More to the point erratic factor index methodologies are also go against the very foundations of factor investing, where the academic evidence points to long-term rewards for those that follow any given factor. “Index methodologies that frequently change factor definitions, or that change the set of factors at different points in time, are inconsistent with the principles of factor investing,” Amenc says.
Welcome to the zoo
Turning to how the most common forms of change to an index, the team identify three forms – factor selection, factor definitions and portfolio construction principles.
As was discussed in a separate piece of Scientific Beta research recently, the academically tested set of factors is both small and stable. They point to the range of factors first identified by Fama and French in 1993 which initially consisted of just three. In the course of the next two decades, just two more factors have been added to their menu.
However, others are not so shy of accepting new factors – hence the term the “factor zoo”. And when it comes to multi-factors, the analysts say they have observed a “fast pace of change over time” with the views of providers views on what constitutes a robust factor “changing dynamically over time”, and sometimes in very short time periods.
“Such short-term variations in fundamental beliefs about factors appears to be inconsistent with the idea that factor indices should represent strategic choices for long-term investing.”
Inconsistency over time will have a “tremendous impact on backtest performance”. And the analysts then provide the numbers to prove their point, showing that factor-picking “inflates backtest results”.
The second route to change comes through factor definitions which, they suggest, providers tend to update “frequently”. Though with academically rigorous research, factor definitions are “very stable over time”, there is a stark contrast” with the factor definition used by providers which often resort to updates.
To illustrate this, they look at how MSCI has evolved the definition of value over time. Of the variables included within the definition, only one – price-to-book – is consistent. All the others change over time with some, such as enterprise-value-to-cashflow, being introduced at one pint only to disappear at a later date. As Amenc et al suggest, “one may ask what the value of such transient variables is.”
The last form of changing definitions comes with portfolio construction principles. When it comes to smart beta, these can change dramatically over time.
“One of the most evident examples in recent years is the increased popularity of so-called bottom- up approaches in multi-factor investing,” the analysts write. “The bottom-up approach builds a multi-factor portfolio in a single pass by choosing and/or weighting securities by a composite measure of multi-factor exposures, as opposed to a more traditional top-down approach that assembles multi-factor portfolios by combining distinct sleeves for each factor.”
This is very academic, but as the team at Scientific Beta has explained previously, there are significant differences between the two approaches, relying as they do on a different set of investment beliefs and objectives.
As they say, bottom-up approaches try to increase the overall factor exposure, accounting for fine-grain differences in composite exposures. “The underlying investment belief is often an assumption that there is a deterministic link between factor exposures and stock returns.”
Top-down approaches, meanwhile, prioritise portfolio diversification over precision in engineering factor exposures. “The underlying investment belief is that expected returns at the stock-level are highly noisy and the relation between factor exposures and returns can only be expected to hold in a broad sense.”
In an understated way, Amenc et al point out that investors “may expect that the fundamental investment beliefs about factor investing of a given provider do not change frequently over time.
“An abrupt change in positioning with respect to bottom-up or top-down portfolio construction thus appears to be inconsistent with sound long-term investing,” the analysts write. “However, index providers have displayed a large degree of flexibility on investment beliefs concerning bottom-up and top-down portfolio construction.”
Shall we tell the investors?
Lastly, the Scientific Beta team turn to how much investors are aware of what might be going on with factor investing as far as what they are being told by providers. If changes are going to be made – and the evidence they provide is that this will happen a lot – then the team suggest that at the very least “changes in index methodology or index offerings should be transparent to allow investors to assess both the reasons for and the implications of methodological changes.”
The Big Call: Smart Beta performance and more!
This is not necessarily the case and levels of transparency across providers are yet another variable. “The question of the governance of index changes should be a subject of concern not only for index providers but also, and especially, for investors,” the Scientific Beta team argue.
“When index changes are announced, investors should get clear information on the details of these changes without requiring investors to do the detective work of comparing different versions of ground rules documents. Providers should also be transparent about the motivations behind index changes,” the analysts suggest.
