George Kirrin’s other famous five

One of the most interesting (and challenging) parts of running a blog is in encouraging and moderating comments. This site gets quite few (real) comments, but sometimes the comments are as insightful as the blogs they comment on. Such has been the case with the comments of George Kirrin and Derek Scott on recent articles from Chris Sier.

George’s challenge to Chris focuses on a credo of data science which Chris endorses. Chris argues that by not measuring the true cost of asset management, we risk not managing those costs. Recent blogs on the work of Ludovik Phalippou have argued that Private Equity carries just such risks , as this tweet explains.

PE is a sham but both investors and the managers are happy for it to continue. Managers because of the egregious high fees, the investors because they don’t have to mark the investment to market like they would a public equity investment, they can obfuscate their own performance!

— Eugen Neagu (@BespokeFS) June 16, 2020

George Kirrin’s criticism of the measurement/manage adage is different and more fundamental.

Out of many received wisdoms “what gets measured gets managed” or Chris’s version that “you can’t manage what you don’t measure” must be one of the more widely accepted as obvious. After all, how could we ever manage something if it isn’t being measured?

The quote is usually attributed to Peter Drucker, the late management theorist.

A bit of research reveals, however, two surprising things.

One, Drucker — according to The Drucker Institute even — never said it. The usual source given by others is Drucker’s 1954 book, The Practice of Management, but I’ve got a Kindle edition and searches for the term (and Chris’s version) both came up blank. I tend to be skeptical of all quotes attributed to great people unless I can find a written source. There are helpful websites out there like quoteinvestigator.com which should be used more often.

Let that penny drop. Various Harvard Business Review articles attributing the quote to him? Wrong..

Two, the fact that something is wrong with “what gets measured gets managed” has been argued for a long time.

VF Ridgway published a paper in 1956 criticising the measurement mantra.

Simon Caulkin, a Guardian columnist, neatly summarised Ridgway’s argument as:
“What gets measured gets managed — even when it’s pointless to measure and manage it, and even if it harms the purpose of the organisation to do so”.

https://www.theguardian.com/business/2008/feb/10/businesscomment1 goes further and asserts that “What gets measured gets managed – so be sure you have the right measures, because the wrong ones kill.”

Ridgway’s 8-page paper is entitled “Dysfunctional Consequences of Performance Measurements”.

It’s common sense that not everything that matters can be measured. It also follows that not everything we can measure matters.

The tendency to report against certain metrics may even distort our priorities.

Pensions consultants tend to measure pension deficits using metrics which Jon Spain, Iain Clacher, Con Keating and others have shown up to be flawed. Other metrics are available, but for some reasons many pensions consultants choose not to show them.

To summarise: Drucker never said the infamous sentence, and since 1956, if not earlier, we’ve been warned that a “what gets measured gets managed” mantra is deeply flawed, Chris.

I will come to Chris’s defence here, because Chris has spent his life measuring the fractional costs of asset management without trying to prejudice wider debates to which George alludes.

Chris would argue that though he collaborates with Keating and Clacher, his work on the cost and charges templates which have allowed us to see and compare costs, has informed on “value for money”. I sympathize with Chris who is not about establishing value, just about money.

However, George’s criticism becomes real when Chris starts validating value in the products he is analyzing. Here Chris comes dangerously close to validating LDI in terms that echo Eugen Neaugu’s criticism of investor’s “obfuscating their own performance”.

And therein lies the point George: LDI, whether it’s reality of fiction, allows you to remove the volatility of valuation, further reducing the risk to the scheme sponsor. At least that is one hypothesis I’m considering.

Measuring should not mean validating

The act of measuring costs and charges within born by a pension scheme in executing investment strategies is a matter of standalone fact. If you can’t find out where Private Equity made its money then PE could be a sham, sooner or later the impact of those costs will work its way through to valuations. Which is why the work of Phalippou is important. The work of Chris Sier on LDI is important, but knowing the true cost of LDI doesn’t validate LDI (as Kirrin points out) anymore than knowing what Phalippou knows , endorses Private Equity.

Here the difficulty is similar to that observed in the NHS by Simon Caulkin

‘It seems unlikely that hospitals deliberately set out to decrease survival rates. What is more likely is that in response to competitive pressures on costs, hospitals cut services that affected [heart-attack] mortality rates, which were unobserved, in order to increase other activities which buyers could better observe.’

If by over-relying on the adage “what gets measured , gets managed” we introduce a false measure by which to judge management, we can accidentally do great harm (Calkin and Kirrin’s point).

Those, including Chris Sier, who see transparency as an end in itself are right to, in the context of the business that they run. But an over-reliance on “what gets measured gets managed” risks not just validating the wrong things but failing to measure the right things.

It is of course a nice irony that the adage itself is of dubious provenance , suggesting a phrase or opinion that is overused and betrays a lack of original thought. We may well think that “what gets measured , gets managed” is a cliche.

Further reading

You can read all Chris Sier’s articles in one place here. If you can’t access the link, then you should register to Pension Expert which is free. Chris has a new blog out asking whether paying more delivers more performance, you can read it here.

You can see all the comments mentioned in the right hand side bar of the blog or by searching “Chris Sier” and clicking through to the comments on each of his recent articles.