Feature Article Archive

Rolling vs. Fixed Period Returns
No Clear Winners

Many investment models are constructed using single point statistics such as five-year return or ten-year alpha. This data is readily accessible and easily understood. Some analysts prefer to measure the same factors over rolling periods. Each approach has its plusses and minuses, and each has its staunch believers. But which is more predictive of future performance?

Historical Differences

Single period statistics measure data over a specified time period. For example, a five-year return is simply the total return from the beginning of the period to a point five years beyond. It’s the return an investor would have received (excluding taxes and trading costs) by purchasing the security and holding it for the next five full years. That’s pretty straightforward but may not tell the whole story.

Consider two investments, A and B, both purchased for $10 and both sold five years later for $20. Both have a five-year return of 100%. But what does this tell you about them now? Suppose A chugged steadily along over the past five years, appreciating on a more or less constant path. On the other hand, B is considerably more volatile, jumping to $30 in the first year but then declining at an increasing rate over the remaining four. Both had the same period return (+100%, or 14.9% annualized), but they took significantly different paths (Chart 1). B only posted a gain in the first year and has been falling more and more rapidly ever since. A, on the other hand, has been a steady performer. You wouldn’t know that from the single point returns.

Rolling returns paint quite a different picture in this example. Rather than a single return spanning the entire time frame, rolling returns divide the five years into equal but smaller periods. The rolling return is the average of the smaller period returns. For example, consider rolling 12-month periods over the same five years. To get this, you’d start by calculating the single point return over the first twelve months. Next, you’d calculate the single point return for months 2-13. You’d continue the process by adding the next month and dropping the oldest month until you had single point results for all forty-five 12-month periods in the five years. The average of these results is the rolling 12-month return, sometimes called the “rolling 1 in 5.” This gives you a good representation of the average 12-month return over the five years. In the current example, Investment A has a rolling 1 in 5 return of 14.9% while B’s is 10.8%. Notice that A’s is equal to the annualized single point return while B’s is substantially less. That’s understandable given that A moved consistently higher while B posted big gains only in the first twelve months before suffering losses in the remainder of the period. In this case, the rolling period results provide a more accurate picture of the relative performance over this five-year period.

But rolling period statistics have their limitations, too. By their very nature, they tend to put more weight on returns in the middle of the measurement period and less on those at either end. It’s not hard to see why. Consider month 1 in the calculation described above. It will only be included in the first 12-month rolling period calculation. The same is true for month 60. Months 2 and 59 will only be included in two calculations. On the other hand, months 12-49 will be included in twelve. Chart 2 shows the number of uses for each of the 60 months¹.

For volatile investments like B, this can have a major impact. Consider two other investments, C and D, which have the same monthly returns as Investment B. However, C’s best monthly returns all occur in the first and last months while its poorest fall in the middle. D’s best monthly returns all occur in the middle of the period. Its worst are at the beginning and end. As shown on Chart 3, both investments still climb from $10 to $20 by the end of the period but they’re mirror images of one another. For both, the annualized single point return is 14.9%, just as it was for A and B. But C's and D’s rolling 1 in 5 returns are significantly different, -1.2% and +35.6%, respectively. Investment D clearly benefits from having its best returns in the middle of the period. Based on these results alone, you might prefer D to C, but take another look at Chart 3. Over the past six months, C has been on a sharp upward trend, gaining 68% while D has lost 10%. Again, this is a function of their volatile nature, but you wouldn’t know this from the rolling returns. In this case, the rolling period results arguably do not provide a more accurate picture of the relative performance over this five-year period.

The Case for Predictive Value

So neither is a perfect measure of investment valuation, but is one approach superior to the other in predicting returns? Or could a combination of the two be the answer? With these questions in mind, we considered the returns of domestic equity mutual funds for the period January 1, 1997 through December 31, 2006. This spanned the tech-inspired equity runup of the late 1990s as well as the subsequent three-year bear market that welcomed in the current decade. It ends with the 2003-2006 recovery.

The study focused on data from two periods: the five years ending December 31, 2001 and the five years ending December 31, 2006. Using the 2001 data set, we divided the domestic equity funds into the nine Morningstar style and capitalization categories. Then we divided the funds into their respective quintiles based on the following four statistics: five-year single point returns, rolling 1 in 5 returns, rolling 3 in 5 returns, and the combination single point and 1 in 5 rolling. To assess the predictive value of these statistics, we then calculated the average five-year returns for each of the groupings using the 2006 data set. The top quintile results are shown in Table 1.

The five-year single point dominated in seven of the nine categories. The exceptions were Large Cap Value (Combined) and Mid Cap Blend (Rolling 3 in 5). Rolling 3 in 5 fared the worst with the combined single point and rolling 1 in 5 only slightly better.

Oddly enough, in any category and for any of the four statistics, the top quintile from 2001 had the highest 2006 return only 6% of the time. For 39% and 31% of the time, the top 2006 return was found in the second and third 2001 quintiles, respectively. Although the five-year single point dominated in the first quintile, it provided the best return in its own group in only two of the nine categories (Large Cap Growth and Mid Cap Growth). This suggests what you probably already suspected: Regardless of how it is calculated, historical return alone is not a good predictor of future return.

What About Consistency?

After finding virtually no predictive value in historical returns, we looked at consistency. One of the intuitive benefits of rolling period over single point statistics is the notion that the former not only measures return but consistency as well. (Recall the different valuations for Investments A and B in the example above.) One might think the combination of the five-year single point and rolling 1 in 5 would provide an enhanced measure of consistency. This, however, was not supported by the 2006 return data. In fact, no factor had a clear advantage.

We used 2006 five-year batting average to measure consistency. A fund’s batting average is the percentage of time it beats its benchmark. It’s calculated by dividing the number of months in which the fund beat the benchmark by the total number of months in the period². The five-year single point top quintile yielded the highest batting average in four categories (Growth for all capitalizations and Small Cap Value) while rolling 3 in 5 was on top in three (Large and Mid Cap Blend and Mid Cap Value). Rolling 1 in 5 had the highest batting average in Small Cap Blend, and the combination of the five-year single point and rolling 1 in 5 was best in Large Cap Value. For all groups, the top quintile had the highest batting average 42% of the time.

Batting Average and Future Returns

We ran a similar analysis using five-year batting average statistics from 2001 as the basis for the quintiles. Again, the funds were divided into quintiles based on their five-year single point, rolling 1 in 5, rolling 3 in 5, and combined single point and rolling 1 in 5 batting averages. To look at predictive value, the 2006 returns for each quintile were then compared, again focusing on the top quintile for each group and category. The top quintile results are shown in Table 2.

This time, the five-year single point group was highest in six of the nine categories, with Small Cap Blend and Large Cap Growth going to rolling 1 in 5 while rolling 3 in 5 took Mid Cap Blend. Once again, the top 2001 quintiles only had the top 2006 returns in two of the 36 groups.

Interestingly, the top quintiles based on 2001 batting average had superior 2006 average returns to the corresponding quintiles based on returns in seven of the nine categories. The differences ranged from 0.03% (Mid Cap Blend) to 1.81% (Mid Cap Value) with an average difference of 0.87%. The two categories where batting average didn’t dominate (Large Cap Growth and Small Cap Growth) differed by only 0.28% and 0.02%, respectively. So, in addition to being a better predictor of returns, rolling batting average was also a better predictor of future five-year batting average³.

Conclusions

For both returns and batting averages, the top historical five-year single point quintiles appeared to be the better predictors of future five-year return. Even so, relative to the other quintiles, they had the highest average 2006 returns just over 5% of the time. This suggests that neither historical returns nor batting averages alone are consistent indicators of superior future return. In this particular study, batting averages enjoyed a slight edge.

Of course, these results come from only one ten-year sample, one that is not necessarily indicative of similar periods. The historical measurement period (January 1997 – December 2001) spans two extreme market conditions whereas the 2006 returns come from a homogeneously bullish period. More comprehensive studies might yield significantly different results.

We can, however, conclude that predictive models should not be solely based on one return or batting average statistic. This holds true for both single point and rolling period statistics. Yet this doesn’t mean that they should not be part of more comprehensive models. Arguably, because they do display some predictive tendencies, they should be considered in effective multifactor models.

Because none of these statistics were clearly superior to the others, the choice of which to use is up to you. You might want to base your decision on your knowledge of the data at hand. For instance, if the funds moved in a relatively steady trend (like Investment A in the earlier example), single period statistics would be appropriate. On the other hand, if returns were volatile (like Investment B), rolling period statistics might more accurately capture the average results. If the best or worst returns were clustered in the middle of the period (like Investments C and D), single period statistics may be more reliable.

Finally, in those situations where you aren’t aware of the return pattern (which will probably be true for the vast majority of your fund evaluations), our data suggests either single period or rolling 1 in 5 statistics will probably be sufficient. Because there was no noticeable advantage in using the two together, this should probably be avoided to minimize the threat of multicollinearity which occurs when two or more variables are correlated with one another. The best factor models are based on factors that are highly correlated with the desired characteristics of the funds you seek but are minimally correlated with each other. Two measures of return covering the same period are likely to be highly correlated, so lacking a compelling reason to the contrary; using both in the same model should generally be avoided.

¹ For a more complete discussion of this effect, see Robert L. Padgette and Timothy D. Paulin, "Rolling Period Analysis -- A Cautionary Piece." July 2006 at www.kleindecisions.com/pdfs/Rolling_Period_Analysis.pdf.

²Just as a baseball player's batting average makes no distinction between singles and home runs, batting average in this context makes no distinction in magnitude. It doesn't matter if the investment's return surpasses the benchmark by 10% or .01%; it still counts as a "hit." Batting average is a measure of frequency, not magnitude.

³Recent research suggests investors may not necessarily want to seek funds or managers with above-average batting average. See, for example, Neil Constable and Jeremy Armitage, "Information Ratios and Batting Averages." Financial Analysts Journal, May/June 2006, pp. 24-34.