‘Mental models’ are the key to measuring the things that matter

What is a mental model? That’s a great question, anonymous website visitor. Mental models are conceptual frameworks that allow humans to make sense of a complicated universe. Supply and demand, for example, is a 300-year-old mental model that helps analysts understand the global economy. Every profession has its own set of mental models. Cognitive psychologists have stimulus and response, bankers talk of bull markets and bear markets, politicians pick ideologies from the left wing or the right-wing.

The most famous mental models in marketing are the purchase funnel and the four Ps. These frameworks help us understand and influence buying behavior. It’s important to note that mental models are, by definition, simplifications. The purchase funnel is not a 100% accurate representation of how consumers buy. But it’s still a useful tool that has helped millions of marketers make faster and better decisions.

To quote the famous statistician, George Box, “all models are wrong, but some are useful”. Which brings us to today’s topic: measurement.

Measurement can be something of a shit-show in our industry. Many of us in B2B (and B2C) marketing have scant idea what to measure, let alone how to measure it. The root cause of this problem, in our humble opinion, is the conspicuous absence of mental models. We don’t know how to think about measurement, so we end up measuring the wrong metrics, in the wrong ways, at the wrong times.

So, what are some useful mental models that can help you make better measurement decisions? Well, we would like to submit three concepts for your consideration.

1. Marketing inputs vs financial outputs

If B2B marketers want to be taken more seriously by their peers, then we need to become much more financial. Money is the language of business, and if you don’t speak it, you will be exiled from the commanding heights of your company.

Every single one of your marketing metrics needs to be tied to a financial metric. If you cannot find empirical evidence from an unbiased source that proves X marketing input will increase Y financial output, then it needs to get deleted out of your dashboard.

Perhaps the most commonly used metric in digital marketing is click-through rate. But click-through rate does not correlate with any financial metrics – in fact, it doesn’t even correlate with any marketing metrics. For over 30 years, researchers have proven (and re-proven) that the ad that gets the most clicks is almost never the ad that causes the greatest lifts in awareness or conversions. Every time you measure your click-through rate, a marketing effectiveness angel bursts into flames.

Compare CTR with a metric like ‘excess share of voice’, also known as eSOV, often measured as your share of advertising expenditure in the category. For over 40 years, academic researchers have shown that increasing your eSOV (a marketing metric) can increase your market share (a financial metric). That’s a CFO-friendly metric – and those are the only metrics that matter.

2. Probabilistic vs deterministic

Let’s talk about the 2016 Presidential Election in the United States, everyone’s favorite topic of conversation. After Donald Trump’s surprise victory, the public heaped endless scorn on pollsters like Nate Silver, who ‘got it wrong’. Except that Nate Silver did not get it wrong. His model predicted that Donald Trump had a 29% chance of beating Hillary Clinton. In other words, he predicted that it was a possible but improbable event. The fact that the improbable event occurred doesn’t mean he was wrong.

The point is that all measurement is probabilistic, not deterministic. Whether you are evaluating the past or predicting the future, you need to ‘think in ranges’, not in precise numbers. Every marketer wants to know the exact ROI of their advertising campaigns. But that’s a wild goose chase. There are too many variables influencing the outcome – the state of the economy, the quality of your creative, the activities of your competitors. And many of those variables are beyond your control.

So don’t say: “We expect this campaign to generate $102m in sales.” Say: “We expect this campaign to generate somewhere between $50m and $150m in sales, and here’s a list of assumptions underpinning the model.” Broad ranges give you many more opportunities to be correct. All good modellers, from particle physicists to risk managers, account for uncertainty in their predicitions.

All marketing metrics are imprecise. You will never know your exact eSOV, for example, because there is no perfect data set on advertising expenditures. But you can still cobble together an imprecise-but-useful estimate of your eSOV, and that will serve you much better than metrics that are precise but useless – like click-through rates.

3. Big data vs long data

Does more data lead to better decisions, or worse decisions? In a famous study, Professor Paul Slovic gave a group of professional gamblers more and more data, and measured the accuracy of their bets. What he found was that having some data is generally better than having no data. But after a certain point, giving a gambler more data will actually decrease the accuracy of their bets, not increase it.

Why? Because of the ‘signal to noise ratio’. In any data set, there is signal (important information that you must heed) and noise (meaningless, distracting information). And as a general rule, more data means more noise, not more signal. So what’s the solution?  The solution is to prioritise ‘long data’ over ‘big data’.

Time is what separates signal from noise. Whether the marketing metric is eSOV, click-through rate or cost per lead, if you measure over a longer time period, you will get more accurate information. Check your metrics once a quarter – or better yet, once a year, and you will be less tempted to react to noise. According to a LinkedIn survey, 75% of B2B marketers optimise their campaigns within the first two weeks. Those marketers are almost certainly reacting to noise, not signals, and hampering their own effectiveness as a result.

Don’t be a marketing day-trader. Be a long-term investor, like Warren Buffett.

More mental models = better measurement decisions

There is no one perfect metric or perfect mental model. In reality you need a basket of metrics that measure reach, distinctiveness and mental availability. And you need a latticework of mental models to identify good metrics and eliminate bad ones.

But here are three simple changes you can make today:

• Measure the right metrics: Connect marketing inputs, like eSOV, to financial outputs, like market share.
• Measure in the right ways: Think in broad ranges, not precise numbers, to account for uncertainty.
• Measure at the right times: Track metrics over long periods of time to separate signal from noise.

Peter Weinberg and Jon Lombardo are global leads for The B2B Institute at LinkedIn.