There is a crack in everything, that’s how the light gets in

Investment performance has become a taboo – a “no go zone”. This suits the auto-enrolment agenda which is “comply or die”, but NEST has broken ranks and asked defaqto to publish comparative investment performance and it was right to do so. It has opened a debate that I intend to pursue.

Yesterday’s information about the actual performance of workplace pension providers (taken from the defaqto/NEST survey) has prompted a strong response from several people, mainly complaining about the difficulty of predicting the future by looking at the past. I know no other way of looking to the future – other than a crystal ball

One of my regular contributors, Con Keating was prepared to look at the numbers provided by defaqto and suggest that they are telling us quite a lot, but not what defaqto thought they were telling us!

Con objects to the method being used to calculate the risk adjusted numbers; to use Con’s peculiar language , he wants us to understand the difference between an arithmetic or additive process and a geometric or multiplisic process (this is – apparently –  known as a log transformation).

To return to planet earth, he reckons that the calculations done by defaqto do not properly account for the impact of volatility and therefore do too much credit to the riskier providers and do not give enough credit to those who are providing returns without risk.

Now I won’t bullshit you, I don’t understand this table but I will tell you that the numbers are straight from Con and what I want to know is whether they are right.

What Con is saying is that despite the excellent numbers achieved in actual terms by LGIM (as an example) the risks LGIM has taken to get to close parity with NEST, provide a threat to future returns. Con estimates that there is a 1 in a 1000 chance of LGIM keeping pace with NEST in the wrong term, because the risks identified in the performance numbers predict nearly 6.5% underperformance from LGIM and 7.5% outperformance by NEST (taken from the highlighted yellow numbers).

There are enough statistically aware people reading this blog to know the difference between additive and multiplistic. They might even be able to check Con’s numbers from the raw data.

Defaqto had turned these numbers into risk-adjusted returns which looked like this

Clearly the results contradict Con’s rankings, who is right and why?

What’s the point in arguing over numbers?

The point is this. Nobody is asking the fundamental value for money questions of the workplace pension providers  based on the facts. Instead the IGCs are chasing rainbows with ever more elaborate surveys of members , establishing what members want.

Members want good outcomes and “good outcomes” means lots of money in later life. They may be able to define how they want that money and they may point to features of a good pension plan (like portals and dashboards and modelling tools), but ultimately members want the certainty of a good pension – for their money.

We need to be able to use actual performance to predict what is going to happen. Data is what statisticians present, actuaries interpret data intelligently, the public decides.

Totally contradictory information doesn’t help

Now – as you might have worked out by now. If you look at these numbers using a geometric return you get one set of conclusions and if you look at them through an arithmetic lens you get another. The most extreme divergence is LGIM which is either at the top or the bottom the least divergence is NEST, which is top dog whichever way you look at things.

The average person doesn’t do stats to this degree and want a reliable means of seeing not just the returns on his or her money, but an analysis on how those returns were arrived at. They want to know whether the good returns were flukes or earned through skill. Con’s analysis suggests that LGIM are fluking it and NEST are getting real value for money.

We need to have a standard way of calculating the risk adjusted returns , relative to each workplace provider, of each workplace provider. The IGCs and the trustees of the master trusts should be agreeing this methodology between themselves and testing it with the FCA.

ARE THEY?

Investment performance is all that counts – so why so shy?

If the IGCs and MT boards are to be relevant, they cannot shy away from these hard comparison. The IGCs and MTs must be prepared to accept that the strategies of their provider is failing against its comparators and against a commonly established benchmark.

This should not be seen as a failure of the IGC or MT. But here I come to my central concern about the governance structures of both. So long as the MT and IGC boards are concerned with the reactions of the insurer or master trust owner to what they do and what they say, they will not put the reputation of the provider at risk.

I note that neither NOW or People’s Pension are included in the survey, Defaqto say they now have numbers from People’s but nothing from NOW, if these numbers are being deliberately with held, we need to know for what (good) reason,

My suspicion is that rather than be held up to common scrutiny, providers would prefer not to co-operate in a proper comparison. This suggests that a proper comparison with an agreed methodology is absolutely the right thing to do.

Three years is not long enough to have proper data, but it is long enough to be able to see trends and for serious questions to be asked about what is going right and wrong.

That is why I and my colleagues are looking into ways of creating a common framework for the IGCs and MTs to judge the investment propositions of their providers by. We don’t want to do this in isolation, we want the industry to come together and agree a single method for doing this and to abide by its findings.

That is transparency in action – not in words.

That’s how the light gets in