Top down or bottom up? How do we assess the value we are getting for our money?

 

Later this week, Britain gets its first ever conference focused entirely on value for money. I’ll be speaking on a panel with Chris Sier who runs ClearGlass (but also helped found AgeWage).

Chris’ work looks at the money side of the equation and helps people work out VFM by giving them a clear view of what they are paying for value. AgeWage looks at the size of a pension pot and establishes whether the money that has been invested to get there, represents good value.

I’m excited by the prospect and in advance, I have produced this paper which explains how AgeWage’s work can be applied to meet immediate needs of Government, trustees and employers considering whether and how to consolidate their workplace pensions.

If you would like to be a part of  this conference, then you can apply using this link. Details are at the bottom of this blog.


The conventional approach is top down.

The DWP want to introduce measures that allow the stakeholders of occupational DC schemes to measure whether consolidation is in member’s interest. This requires an assessment of the value for money of both the ceding and acquiring schemes. The FCA have been consulting on a single definition of value for money.

In the past, such assessments were carried out by testing the “money” element of each scheme. Member detriment was identified only when explicit charges became higher as a result of the consolidation. This is still the case in many value assessments.

In its consultation on improving member outcomes in September 2020, the DWP introduced a broader approach to VFM which measures both value and money and have proposed a top-down approach to assessing value based on “net performance”. Past performance is analysed by applying the scheme’s actual charges to actual performance, providing a proxy for the average return a member would have got. I call this approach top-down and it will remain the conventional solution for some time to come.

This approach is good as it allows for a separate analysis of the costs and charges and performance of the scheme. The Pensions Regulator has identified 32 qualities of a good DC scheme and a VFM analysis could create a top-down balanced scorecard on all 32. The DWP’s simplification of net performance simplifies matters and it has my full support.

If the trustees of a smaller scheme consider that the scheme is not delivering good overall value following its value for members assessment, Government expects trustees to wind up the scheme and consolidate.


A bottom-up alternative

The top-down approach has limitations for DC schemes. It relies on assumptions about how charges and performance feed through to the member’s pots but cannot capture the experience of the member which is measured by the internal rate of return on their individual pots. There are many aspects of the DC experience, such as the costs of buying and selling units during lifestyle and the impact of the sequencing of contributions and drawdowns which can’t be captured by net performance.

We have urged the Government to look beyond this top-down approach and consider value for money at a more granular level, measuring the actual return that members have received, member by member. There are three advantages of this method over “net performance”.

  1. It is outcome based and shows the actual impact of all risk and reward to the member’s pot.
  2. It is readily benchmarked against any index showing the average return on a member’s pot.
  3. It throws up data anomalies (outliers) where the internal rates of return on a member’s pot are unfeasible.

We believe that this “comparative IRR approach” is the right direction of travel for VFM measurement and we are encouraged by the increasing airtime this new science is getting. We do not expect it to supplant net performance, but we see it being used alongside it, especially where all or parts of larger schemes are being consolidated.


Why use both?

It is clear that the DWP has established net performance as the standard way of comparing value for money and to suggest a second method, where internal rates of return are compared, may seem overkill. But where a decision is being taken to transfer all or part of a scheme’s membership on an irrevocable basis, both acquirer and the ceding scheme will want to ensure that members are properly cared for. This duty of care – or fiduciary duty – sits primarily with the trustee but extends to the employer where reputational risks are being considered.

It is also important – commercially – that both parties are aware of the quality of the transfer, both in terms of achieved value and in terms of the quality of data (a good proxy for the quality of service) being transferred.

The ceding scheme will want to be clear that whoever acquires their members will have history providing comparable value to members and will also want to protect their members from situations that might emerge where a scheme is damaged or even wound up for poor quality data. Similarly the acquiring scheme will be keen to know the quality of data it is acquiring and the value that has been achieved (especially if their scheme will have to accommodate poor value and data). The attraction of prospective assets and data under management improves with transparency and a proper data and performance assessment (including risk adjusted measures) should be critical in determining the term sheet.

We suspect that much of the consolidation happening at present is not so much based on matters impacting the member’s outcomes as on tangential measures which often add up to good marketing (but not always much more).

Metrics relating to the actual performance of a scheme based on the experienced outcomes of member’s saving, are rare.


Comparisons

We’re all aware of the dictum “If it doesn’t get measured, it doesn’t matter”, but if the measurement can’t be calibrated against a benchmark, it is abstract and inapplicable.

We believe that both performance and data quality can be compared if triangulated by a comparator. In the case of data quality, the comparator is the perfect scheme which has no data anomalies, in the case of performance the comparator is the return experienced by the average saver. An index for the return of the average saver exists – the Morningstar UK pension index.

Below are two examples of how DC schemes can be compared, the data sets include multi-employer and single employer occupational schemes, GPPs and Stakeholder pension schemes.

This chart shows the dispersion of data quality by scheme size, the x axis represents the percentage of member data sets which were “suspect” while the y axis represents the assets under management for each scheme.

This chart shows the dispersion of value scores for the same schemes. The x axis shows which schemes beat 50 (the average) and which didn’t – the score represents the performance of the aggregate of member pots against the benchmark. The y axis represents the size of each scheme.

The charts show how group data sets can be compared. The same method can be used to compare the risk and reward of individual pots. We use an Experian style scoring system (scored out of 100) but there are many other ways to express the comparison.

We acknowledge that though what matters most to members is the size of the pot (the net asset value), there are other things that matter too such as consistency of outcomes. Datasets allow us to see the value gained per unit of risk taken.

Value can be expressed in terms of the return achieved against a variety of benchmarks including cash, the cost of an annuity and a specific benchmark a DC scheme has given itself. But ultimately, we want to see comparisons which can compare one scheme with another. We want to compare a pot within a GPP, a commercial master trust and a single employer DC scheme.


Conclusion

In summary we see clear advantages in both top down and bottom-up assessments of value for money, while we are currently relying on top-down assessments, the bottom up approach of assessing individual IRRs is likely to continue gaining traction. Our hope that this will be recognised in future Government documents including the upcoming response to the FCA’s CP20/9 and the statutory instruments resulting from the DWP’s Improving Member Outcomes consultation.


 

About henry tapper

Founder of the Pension PlayPen,, partner of Stella, father of Olly . I am the Pension Plowman
This entry was posted in pensions and tagged , . Bookmark the permalink.

Leave a Reply