A breakthrough in measuring the quality of DC scheme data.


Protecting members from poor administration is a key fiduciary objective

One of the critical issues facing Government in establishing a pensions dashboard is “data readiness”. Currently it is in a relatively weak position to argue that schemes are or aren’t dashboard ready, since it has no way either to assess the quality of scheme’s data-readiness or to assist schemes in self-assessment.

The problem

The reasons why Defined Contribution data might be wrong are numerous. In most cases, the administrator cannot be held responsible for what goes into the database that keeps the member record.

Contributions are received from employers on behalf of staff and can easily be miscalculated.  Work done by Ros Altmann and PensionSync suggests that there may be a high percentage of marginal miscalculations among SMEs using auto-enrolment. These errors are not a pension problem , they are a reward and payroll problem and should be treated separately

There is nothing that a pension administration team can do but to accept the declaration of payroll that the contributions received are correct, but if an error is made, the bulk of restitution falls to the blameless administrator.

Similarly, the contributions received from the DWP (formerly DSS) by way of protected rights contracting out rebates  had to be taken as right, there being no way of reconciling what came from the DSS Newcastle operation with people’s entitlements.

Even the contributions from HMRC – representing pension relief at source, were typically accepted to be the responsibility of the tax-office and not the pension administrator.

But in some cases, administrators make mistakes, usually where manual processes are used. Examples are in the receipt of one-off contributions , claims and in member generated fund switches (as opposed to automated fund switching – as a part of lifestyle de-risking).

The vast majority of data errors that occur by pension administrators occur in the administration of contributions in and member initiated changes (switches and claims).

It is these errors that can be genuinely called pension mistakes and these that this article will concentrate on.

The current situation

I am writing with the privilege of analyzing over 1,000,000 contribution records held by administrators of UK DC schemes on behalf of trustees and within contract based plans.

It is clear that the quality of data held varies. Our evidence is based on the process we adopt to create AgeWage scores. What we do is to marry the contribution histories and units they have purchased to the eventual unit holdings which determine the pot value. This process is common and allows us to work out an individuals “internal rate of return” or “IRR”.  IRRs are factual but they don’t tell us whether they are right.

To sense check if IRRs are reasonable we have to apply a second test to the data. This requires us to apply the same data to a synthetic benchmark fund – created for us by Morningstar (the Morningstar UK pension index). This fund tracks the progress of the average DC investment on a daily basis going back to 1980 and enables us to work out a synthetic IRR, being the return someone with that same contribution history would have got if they had been an average investor.

For the vast majority of records we look at,  the achieved IRR and the benchmark IRR are close enough for us to validate the data as “making sense”. But in a small proportion of cases, the data does not make sense, here the two IRRs diverge sufficiently for us to consider the divergence unlikely and in some cases very unlikely. We categorize these cases as outliers and  when we deliver our reports to the holders of these data sets, we list the outliers and determine their rate as a percentage of the data set.

Data showing outliers where Actual and Benchmark IRRs exceed set tolerances

We have seen outlier rates as low as 0.2% of data and as high as 8% of data. Typically, the outlier rate is around 3%.

The application of this work

By treating data in this way, we can show trustees, employers, providers and eventually members, where there are potential problems with data. Hopefully errors can be fixed before they are admitted to members and this process – known as cleansing – can be carried out relatively easily, once “outliers” have been identified. I say “relatively” as most errors aren’t as easy to fix as to identify but failures in identification has – in the past – stopped many schemes getting to first base.

It would be possible for schemes, using this methodology, to self-assess their data and establish how dashboard ready they were. It would be helpful if Government could validate the process and make it available to pension data administrators. The results of such self-assessment would have some immediate advantages

  1. Those people in DC schemes approaching the point when they wanted their money back could be assured that the money in their pot was “right”.
  2. The pension administrators could feel comfortable that their liabilities for errors were mitigated by this early warning system
  3. The trustees and provider IGCs could speak with authority in their chair statements about the quality of service being offered ( a component of the value for money assessment)
  4. Government could have a clear view of the capacity of schemes to integrate with the pensions dashboard with sensible results for those viewing their data
  5. The Government’s regulators could have proper information on data quality and be able to manage situations where schemes were failing , more accurately and earlier.

Next steps

AgeWage makes this offer to Government and to the pension administration industry. We have found a way to process data in bulk and provide what we consider reliable metrics on data quality and pull out “outliers” for inspection and potential data cleansing.

We welcome feedback on this idea either as comments on this blog or directly to its author henry@agewage.com


About henry tapper

Founder of the Pension PlayPen,, partner of Stella, father of Olly . I am the Pension Plowman
This entry was posted in advice gap, age wage, Dashboard, dc pensions, digital and tagged , , , , , , , , . Bookmark the permalink.

3 Responses to A breakthrough in measuring the quality of DC scheme data.

  1. Dear Henry
    It is great to see the increasing recognition of pension data errors and calls for reconciliation. The data problems, however, are far more widespread than just ‘outliers’. As you rightly say, where manual processes have been involved for many years, hasic errors in the original salary records, in the calculation of accurate contributions, and even in dates of birth, names and addresses, have resulted in a growing backlog of problems that have never been properly identified, let alone corrected. Even with auto-enrolment, we saw at pensionsync that small firms using manual payroll entries, would even have incorrect tax relief applied due to confusion around Net Pay and Relief at Source. The complexities of pension rules mean that most people cannot know for themselves whether their pension contributions are correct, they rely on their employer or pension company to have the right figures on their behalf. This is very different from ‘open banking’ where you know how much you have paid in to your bank and can see if the amounts shown on your statements are incorrect. If you pay in £20 and the statement says you have £18 in your account, you can see it is wrong. But if you had no idea what you paid in and the statement shows you have £18, you simply will not know it is wrong. That is how pensions are for most people.
    Over the years, and even since 2012 with auto-enrolment, records have been ‘assumed’ to be correct, with no regulatory demands for accuracy checks required, no mystery shopping to select random samples that could pick up errors. Indeed, pension providers often have no idea what each member’s pensionable salary was each month, which means the checking is required at the employer or payroll administration end.
    Yet, it is really vital that pension schemes check all records, rather than just the extreme cases, otherwise the information on the Pensions Dashboard is unlikely to be sufficiently reliable. This exercise could and should have been started years ago, but sadly we still await it.
    When it starts, it will be a huge undertaking and we have seen with GMP reconciliation that the problem magnifies over time, costs huge sums and takes years. The longer it takes to start the process, the longer records remain incorrect.
    I do hope the Pensions Regulator and Government will finally force all schemes to prove they have taken this issue seriously, that annual data checks are made and reported to Regulators to confirm that contributions have been verified and introducing digital connectivity would be a major improvement for auto-enrolment, that can reduce manual errors.
    I did try to get amendments in the Pension Schemes Bill in 2017 and in the current Bill, but these were rejected I’m afraid. If data are not reliable and have not been reconciled, then I can’t see how using a Pensions Dashboard can ultimately ensure consumers ‘know’ what their pension position is.
    You’re absolutely right, schemes need help with data, but the problem is far wider than just ‘outliers’. The extreme cases are much more readily identified, but the small errors compounded over time can also make pension records worrying inaccurate.

    • henry tapper says:

      Ros, we have found that over time , a compounding error leads to an outlier – even if the error is small. You are right to say that the problem is wider than our outlier test can highlight, but you have to start somewhere.

      The problems we see should be throwing red flags to Trustees and IGCs, but – as Bob points out – the issues surrounding the contributions themselves, are falling between two stools with neither employer nor provider, being accountable for what’s fallen!

      The Pension Regulator has little power over DC administration , other than to oversee the contributory aspects of AE. I’m pleased to see that there is talk within the DWP , to test the quality of data and I think firms such as PensionSync and AgeWage are the kind of innovators who can and should help them. This is not going to be solved by the big established players alone!

  2. Bob Ward says:

    I follow the line of Ross Altmann and agree there are wider issues here. However, what is missing from the case is ‘innertia creep’ – a term I use to explain the loss of intentions for integrity of systems and meeting Statutory and Contractual obligations.
    The AE legislation puts the onus fully on Employers for ensuring they comply with the legislation and perhaps you are nearly right in that Administrators / Providers are not repsonsible for Employer / payroll errors.
    However, time and again in our industry, those involved are blinkered to think they only have to comply with the simple pension legislation but that is NOT the case. Master Trusts and contgract based Providers are obliged to declare to tPR that they have checked that particiapting employers are complying with AE rules and if not they must report the Employer. The only way to do that is for ALL providers to sense check contributions against the AE rules and pensionable earnings provided by the Employer (i.e. earnings x AE contribution %).
    When configuring their systems in early AE many providers took the decision it was too difficult to check contributions and hence pushed the assessment back to the Employer’s payroll. That way the Providers have not carried ourt assessment calculations. They therefore may now adopt the stance of accepting the contributions given without rechecking that even the amount meets the above calculation.
    Consequently, Providers are not meeting the AE Codes by not physically checking the Employer figures.
    Innertia sets in with everyone thinking all is ok and that the Employer takes full responsibility and tPR does not seem to be drilling down on verifying exactly what is being checked. If such errors are found in the future the Providers will be equally responsible by not meeting the due diligence standards.

Leave a Reply