How not to forecast: The herd mentality is alive and well

-Donal Curtin

Tuesday 4th September 2007

HERE'S A party trick: I'll put on a blindfold. Meanwhile, you'll ring up the banks and ask their economists for their interest rate and exchange rate forecasts. Don't worry that they won't have any; they produce them all the time. You'll write them down and hang up.

And then I'll tell you what the forecasts are. Doesn't matter what banks you called. Doesn't matter what year. I'll know what they're forecasting.

It's not a difficult trick, for one simple reason. As a group, they make the same forecast, all the time. Sure, it's not the same in a strictly literal sense: it isn't always 8% for interest rates and 65 cents for the currency. But it's the same forecast all the same. And here's what it is.

If the exchange rate, or interest rates, have been going up, the forecast is that they will go up a little more, but then come down. And if the exchange rate, or interest rates have been going down, the forecast is they will go down a little further, and then move up.

Daft, isn't it? Dozens of intelligent, well trained, highly motivated people have been involved in this exercise, and every single time it defaults to the same clunky outcome. It's clunky because it hasn't worked as a successful forecasting method. It's clunky because it's used in all seasons and all weathers when clearly the same answer can't be right all the time. And the daftest thing of all is that, even when this clunky mechanistic way of forecasting has been discovered and published and documented as being no good, everyone involved keeps on doing it anyway.

There had been earlier research showing this pattern, but it was documented again a few weeks ago in Christchurch, at the New Zealand Association of Economists' annual conference, as part of a paper assessing the Reserve Bank's forecasts over the past four years. Right up to the end of last year, the economic forecasting community were still doing the same old, same old when it came to forecasting two of the most important financial influences on business.

It's worrying. For one, you can't rely on the economists for the warning you need if the currency is about to head for years in the same direction, as it did from 1988 to 1992 (downwards), 1992 to 1996 (upwards), 1996 to 2001 (downwards), and from 2001 to date (upwards, though with a temporary backtrack in 2005). They don't do multi-year-trend forecasts, even though that has been the typical pattern for the Kiwi dollar. Ditto for interest rates.

Why could this be? My suspicions are two. One is that very few of the exchange rate and interest rate forecasts are driven by quantitative modelling. Econometric or other techniques aren't the answer to everything, but they do impose a discipline and logic on the process that seat-of-the-pants approaches can't. The Kiwi dollar has more than doubled in value against the yen since 2001, for example: a model that formally links the yen cross rate to interest rate differentials between here (high) and there (exceptionally low) stands a chance of being right. An approach that stays with "it'll move a little higher and then drop back" has no chance.

My other suspicion is there's a degree of herding going on: forecasters are, for whatever reason, uncomfortable being far away from the consensus. It's too risky - to credibility, perhaps - to make an unconventional forecast that might go embarrassingly wrong. You see it all the time in the investment game, where fund managers, for example, are often afraid of being the only one holding a distinctively different portfolio of share picks.

Herding behaviour is especially evident when it comes to forecasting the likes of GDP. The entire forecasting community had too low a forecast of 2003's GDP growth, for example, and they did exactly the same in 2004. There wasn't a single forecaster in either year who had too optimistic a forecast: every single one was in the same over-pessimistic camp.

You could argue there's nothing surprising in that: any group of professionals with broadly the same sets of diagnostic skills will tend to come to similar conclusions faced with the same sets of facts. But it's equally suggestive of everyone clustering in the same, safe place.

It doesn't have to be that way. If you look at the Wall Street Journal's forecasting panel of 60 US forecasters, and their latest forecasts for how fast the US economy will grow in the final quarter of this year, you find a genuine variety of views, ranging from very strong growth of more than 4%, to close-to-recession 1% growth. You're still left with the issue of who to believe, but at least you're being given a menu of independent choices, unlike here.

The bottom line for businesses? Don't place much weight on the economists' exchange rate and interest rate forecasts until they start to do them in a different, less mechanical way. But do pay special attention if one of the economists goes off on a tangent: they mightn't turn out to be right, but at least they'll be exercising their own independent judgement.

Comments from our readers

No comments yet