It's impossible to measure all things equally. And that's a good thing

Are we measuring the right things?

When we first published Making Sense: The Commercial Media Landscape back in late 2019, we had one core objective and that was to provide the industry with a solid foundation of the changes within commercial media consumption. I’d grown tired of the constant industry narratives stating that ‘this media was dying’ and that ‘only media which reaches 16-34s is subscription based and that advertising has no hope’. It is so easy to get caught up in these constant conflicting narratives, that we often forget the main objective of advertising. Ultimately, the reason we are all here is to facilitate the successful growth of businesses by providing means of communicating to a large number of people and to drive brand and business outcomes for clients.

The measurement trap

One of the hopes for the Making Sense work, was that it would allow changes in commercial media consumption to be analysed on a level playing field using a single source dataset - the IPA’s TouchPoints. The report for the first time, looked at reach, share of time, time spent and patterns across the day of all commercial media properties in order to help show where shifts were occurring. In many ways I hoped that by utilising a variety of measurement metrics, this would help to end the dying media debate and move the narrative away from seeing reach as the only metric that mattered.

The reality was not quite what I expected. While established traditional methods of measurement for Print, Radio and TV relied primarily on reach metrics as proxies for engagement and attention, there has been an ongoing debate raging in the industry about how new digital properties are measured. The need for a reflective look at how measurement was understood within the industry came when a comment was made to me about how it was great that Making Sense looked at the share of time which average adults, 16-34s, 35-54s and 55+ spent using each media property as it enabled them to look at share of attention directly and to apportion and justify share of spend. This relied on one problematic assumption – that all impressions are created equal and that direct comparison of different commercial media is not only possible, but also desirable.

As human beings and equally as an industry we often crave a single right answer to a question. Binaries are seen as giving clear direction – If it’s not this, then it must be that, and again that brings us back to the dying media narrative. Often justifications are made along the lines of ‘Broadcast Linear TV Reach is down and Social reach is up, therefore audiences have shifted their attention from TV to Social and investment should reflect this.

But if we look at need-states, media multi-tasking and context at the moment of engagement the story shifts dramatically. For years within TouchPoints we have been tracking the patterns of all Video, Audio, Text and OOH based media across the average day. And guess what, despite new platforms emerging, prior to the first 2020 Lockdown, the patterns of consumption hadn’t changed in 15 years. So while delivery methods may have shifted, consumer need-states remained constant.

Moving away from seeing measurement as just about reach enables a far greater possibility for understanding context, emotional readiness for messaging and equally effectiveness.

Making Sense (3rd edition) showed that in just 6 years, the correlation between the commercial media time by property between an average 16-34-year-old and an average 55+ year-old had dropped from 58% to just 9% evidencing that although they may use many of the same commercial media properties, the way the two audiences use them is vastly different. And this is just a broad example using a rather blunt segmentation of people. Just imagine if we moved away from age as a defining factor of behaviour to motivation, interest or even context. Rory Sutherland once said “don’t plan for the average, that person doesn’t exist” but so often that’s exactly what we do.

Efficiency ≠ Effectiveness

The big problem in how we often look at measurement is that we try to make the old models and understandings fit new circumstances and see the ability of a media to reach people as evidence of its ability to effectively influence their purchasing decisions. I’m sure we’ve all heard narratives saying that you can’t build brands on certain media platforms and often these narratives are rooted in problematic understandings of media. Video here is viewed as video there and you can apply an understanding of measurement of a TV campaign to a YouTube one, but that’s just not the case. Let’s take a common scenario - 60 second TV ad created, objective of brand building, put on YouTube with a 5 second skip capability, metrics show high levels of reach, unprompted brand awareness remains the same – YouTube is seen as not good for brand building. In this example, that’s no surprise – of course the campaign failed as the assumptions from the outset were flawed. This wasn’t planning for effectiveness; it was planning for efficiency.

If we plan for efficiency essentially our goal is to gain the highest possible level of reach, impressions and frequency at any given level of investment and forget brand or business objectives, they won’t figure. Had the campaign metric been 5 second video views instead of just reach, the creative would have been optimised for most people only seeing 5 seconds of the ad. The brand would have been front and centre and unquestionably unprompted brand awareness would have shifted. The big problem with how this would be viewed is that from an efficiency side, the CPM would have been much higher even though the effective outcome is far superior.

Bringing it all back home – Effectiveness is everyone’s responsibility, not just Les Binet’s

The fact that it’s impossible to measure all media on a level playing field is a good thing, as seeing all media impressions as unequal and in different ways helps to ensure that we plan appropriately for the desired outcome. So much time is given to reach based arguments for media, but how much time is given to questioning why? What do clients ultimately want advertising to achieve? – to drive brand and business outcomes – to make them more able to do the things they do so well, provide goods and services to the people that want and need them.

We don’t want a culture of efficiency, but we certainly need a culture of effectiveness. Within the advertising supply chain, effectiveness should be front and centre for everyone. Ultimately, if what you are doing for the client isn’t focussed on making the outcome more effective, it’s probably making it more ineffective. Don’t be afraid of test and learn, embrace case studies and the wealth of data we have within the industry – new media need new methodologies, and as long as the focus is on the desired outcomes, convincing them to try new things will be much easier.

Remember, a client won’t remember how many people the metrics say their campaign reached, but they certainly will remember whether the campaign was effective or not.

Simon Frazier is a Senior Research & Marketing Manager for IPA TouchPoints and recently hosted a Making sense of media measurement event and panel with Shazia Ginai – CEO, Neuro-Insight, Pete Buckley – Connections Planner, Facebook and Helen Rose - Managing Partner (Insight & Analytics), the7stars to discuss the future of measurement and what it means for the advertising industry.

The opinions expressed here are those of the author and were submitted in accordance with the IPA terms and conditions regarding the uploading and contribution of content to the IPA newsletters, IPA website, or other IPA media, and should not be interpreted as representing the opinion of the IPA.