Mark Ritson: Facebook should hang its head in shame for measurement errors

A lot of people ask: “Mark, how long have you been married?” That’s a tricky question for me because I work at Facebook (disclaimer: not really). So rather than come out with a single comparable metric like – oh I don’t know – years of marriage, I prefer to approach the answer differently.

Usually I trot out about a dozen different data points one after the other. These include average number of waking minutes spent with my spouse per day and the total number of consonants exchanged between us during weekly conversations. Crucially, I can then add in how many times we have sex per week and the number of times we exceeded the three-second mark that separates ‘foreplay’ from ‘intercourse’.

It’s a lot of information and the person asking the question is often stunned into silence by the sheer range of data that I provide. Indeed, at this point, most people wander off with a suitably dazed look on their face. If they stick around I then show them some charts and a few interactive animations using my smartphone that capture the current state of my marriage and duration of my most various recent interactions with Mrs Ritson.

Measuring so much data is, of course, very tricky and much of it is (as far as I can currently tell) totally incorrect. But that’s really not the point. The point is I have loads of metrics and a smashing dashboard and, sorry, what was the question again?

READ MORE: Facebook admits more measurement errors

Yes, you guessed it. We are back in the data dunce’s corner once again with Facebook. Last week it made another four admissions of measurement error. Along with September’s announcement, that makes five entirely straightforward calculations gone wrong in as many months and the distinct expectation that there might be more to come.

Once again, we were immediately advised by Facebook that none of the mistaken metrics mattered because they were not used as the basis for pricing or payment. But that, of course, is entirely disingenuous given any incorrect data could have influenced the initial decision to invest with Facebook in the first place, especially when four out of five of the errors were in its favour and by up to 80% over reality.

If TV fessed up to 80% overstatements of reach or frequency it would be vilified.

These latest errors are relatively minor but Facebook is growing like the clappers and will soon form, along with its quieter and less error-prone rival Google, the ‘digital duopoly’ (get used to that phrase). These two companies will account for more than 50% of the total digital advertising revenues in the UK this year, according to eMarketer, which in turn occupies an ever greater majority share of total ad revenues. That means Facebook is the new advertising orthodoxy and, as Confucius famously observed, when you sit at the top of the mountain everyone below tells you to go fuck yourself (or something along those lines).

READ MORE: Mark Ritson – Facebook’s erroneous video metrics show no one has a clue about digital

Just as a young Mark Zuckerberg nervously positioned his formative advertising offer against ‘traditional’ media advertising a decade ago to gain awareness and ultimately market share, everyone with a brain and a rival minority medium is attempting to capitalise on Facebook’s latest empirical misfortune and portray the company as unreliable and unworthy of client business.

The measurement errors also run directly against the big data, empirically-driven modelling that Facebook promised everyone while it was tearing up the traditional rate cards of formerly dominant media channels.

If TV, or God help it the print media, fessed up to 80% overstatements of reach or frequency it would be vilified and the mistake taken as clear evidence of an attempt to hide away its failings and decline.

Facebook deserves the same scorn twice over, because it was meant to be better than all this and, so far, it has been worse. And no-one knows how bad it really is because, no matter how many times it talks third parties, Facebook still has a massive wall between its garden and everyone else’s.

READ MORE: Advertisers must become ‘militant’ over digital metrics

The good news is that the sheer size and growth potential of Facebook, which analysts expect will generate around £900m in UK advertising revenues this year, makes it highly unlikely that clients or agencies will put their head over the media parapet and complain. Instead the usual digital suspects, sockless and supine, obediently tell all and sundry that there is nothing to see here. Move on.

But the big error in all of this is the big data. Facebook is quick to point out that its five mistakes are a tiny fraction of the 220 metrics that the company currently uses to measure audiences. But that is not a statement in its defence, it’s an explanation for its error. I can understand why it might want more than the classic media measures, but 200 more?

Naivety and inexperience on the part of Facebook’s researchers mean it is measuring too much. The skill of audience research is to know both what to measure and what to ignore. Measure too much and you confuse your message, your clients and ultimately – as is becoming increasingly apparent – yourself.