Measure 4 Measure (why we rate things)


A very curious article appeared on the NOW blog yesterday, I took exception to it and wrote to NOW to tell them. You can read the article here. It’s author, the excellent Martin Olive had clearly become frustrated with “rating agencies” -by which he thinks deFaqto.

My objection with the article was that it dealt with an historical anomaly as if it were critical.  IFAs used to get involved in rating workplace pension provider, when there was money in it. There is no money in it- there is no demand for ratings (at least from IFAs) so deFaqto – quite properly- do not regard rating workplace pensions as a very important task. DeFaqto ratings are not going to determine the success of NOW pensions, that will be determined by their performance, and that will be determined by the market sentiment towards them.


Case study; How Pension PlayPen rate NOW.

How can NOW measure market sentiment? Easy- they can listen to their own customers. If they want to compare their UX (trendy for customer experience) they can look at their rating (which is typically very good). Unfortunately NOW have problems with their administration, especially their interfaces with employers. This has seen their ‘integration’ score fall on from 8 to 2 out of ten in recent months.

Let me explain how this measure is composed. It is an aggregate of the UX of a basket of payrolls we survey on a regular basis, these include software houses, bureaux and the larger in-house payrolls. It is an aggregate of feedback that First Actuarial receive from NOW clients (typically larger clients) and it also takes into account the UX of Pension PlayPen clients who have staged using NOW.

Together , the user experience from these sources gives us a tactical view of where NOW’s administration is today. We then take a view of the outlook for that administration improving, this is based on our meetings with NOW.  We have to be frank- unless there is real progress in sorting out a long-term administrative solution for NOW’s problems, our rating for administration will remain low. We continue to monitor.

Why ratings matter

It’s often been observed that if you can’t measure something, you can’t govern it. The measures we put in place to rate workplace pensions are based on what the Pension Regulator regards as the 6 factors that determine good DC outcomes. So our measures are outcome based.

But the need to maximise member outcomes must be balanced with a need for the pension to “work” and work in 2015.

Typically, users of regard the immediate usability of a workplace pension as about 30% of the measure taken in rating the pension with factors relating to outcomes making up the other 70%.

We monitor this balance because it tells us what matters to the purchasers of these pensions, employers and their agents. Employers and agents who take the trouble to compare providers using , leave their digital footprint on our site. The decisions they take influence the decisions of others (employers like you…_

Conviction and integrity

But this “social media” approach to establishing what is important (listening to the likes), is only half the story. Ultimately employers and agents are looking for guidance from an authoritative source. Our authoritative source is First Actuarial, who analyse providers through an analysis of questionnaire responses and through probing on investment issues to get a proper understanding of “value for money’.

We choose First Actuarial because we value their integrity and the conviction with which they rate providers. Being independent of the providers, not taking hospitality is critical to this integrity,  their having the courage to have conviction, critical to the value of their opinion.

The new measurers

As I’ve been at pains to point out, what we rate and how we rate providers depends on what matters to the users. Some ratings will go up and down depending on temporary factors but some things won’t change that often. That’s why we need fundamental ratings based on conviction and integrity of experts as well as the more ephemeral feedback of employers.

Our job, in our ratings is to get the balance right, to be fair and to have courage in our conviction.


NOW should look forward not back

NOW is a new brand, little older than Pension PlayPen. It had the courage to be different and so do we. Looking back to the old way of rating- before RDR, before AE and before the arrival of NOW, Peoples and Nest is regressive.

Pinning your hopes on one measure of governance (the mastertrust assurance framework) is likewise a dangerous game. For NOW, as for all other providers, the ultimate test of value is their user experience. The immediate measure- current employer feedback is one thing, the longer measure- member outcomes – another.

Accepting that they have a short term problem with administration, NOW should be encouraged that our longer term ratings remain positive. If they want to be “top of the pops’ they are going to have to do something about the current UX (and they know that).

And if they want to get properly rated, they should encourage organisations like Financial Sat Nav and Pension PlayPen and not carp that they can’t get properly measured,

Look forward not back

due diligence now

About henry tapper

Founder of the Pension PlayPen,, partner of Stella, father of Olly . I am the Pension Plowman
This entry was posted in consultant, dc pensions, FCA and tagged , , , , . Bookmark the permalink.

Leave a Reply