Measuring the unmeasurable

PR campaigns are increasingly all about creating ‘buzz’. But how can this be monitored? Adam Hill reports

Agencies and their clients knew where they were with rigid media channels – press, radio, TV –  and ever-tidier measures such as column inches, number of mentions, reach, advertising value equivalents and opportunities to see.

Anything outside the traditional media is still seen as difficult to track, measure and analyse. 'Clients ring us saying "I'd like to track the world's media",' laughs one measurement company boss. 'It's impossible.'

The problem is that more and more ephemeral media channels are touted as contributing towards consumers' overall picture of the world, ranging from blogs, viral messaging and word of mouth (WoM)to 'buzz' marketing – or 'talkability'. Clients naturally want to measure these too, but how to achieve that is another matter.

Most of the problems with these techniques, say measurement firms, stem from the fact that viral and WoM originate from the web, much of which is still unmeasurable. Statistics on company mentions or viewings of a particular story on news websites will rarely be forthcoming unless the site itself supplies such data to ABC Electronic.

Even if these statistics were forthcoming, PROs would need to ask readers directly what particular companies they had looked at on any given page, or must assume that all competitor mentions have been viewed too. Figures for page impressions or unique users per month cannot take re-reads into account and the number of people directed to sites by WoM.

Content measurement
Measurement companies are attempting to overcome this in various ways. Millward Brown Precis uses 'content visibility calculations' (a tool that measures how many times a company appears in a block of text to give it a density score, and a measure of how much awareness will be produced) to add an extra dimension to data on page impressions, unique user visits, ad impressions and ad clicks.

However, as online press and promotions agency Way to Blue – which works for clients such as UIP distributing teasers before a film's release – knows, trying to make more of the numbers is not easy.

Publicist Matt Park says: 'We host the video files ourselves and can track the views of trailers, clips and interviews, which tells us how many people read the articles and how many of them choose to interact with the features and find out more.' But, while he knows that the trailer for the film Serenity was viewed more than 100,000 times in the same week that an online cricket game promoting Channel 4's Ashes coverage was played more than 900,000 times, how much traffic came from WoM is difficult to know.

This intangible nature of newer media channels presents another bone of contention – to measure or not? For example, Dan Naylor, director of sales and client services at media monitoring service Durrants, says: 'There doesn't seem to be a great demand for monitoring blogs. This is predominantly because of the lack of control. People rightly say that there might be something negative written about a firm in a blog but there is nothing we can do about it.'
Not everyone agrees. Millward Brown Precis monitors and analyses consumer-generated media (CGM) such as blogs and message boards for clients including BAT, Gillette, Vodafone and Ford, using its Precis:cubed tool, which monitors 5,000 news sites plus 80,000 blogs and bulletin boards.

CEO Fergus Hampton says it allows clients to review the effectiveness of their comms strategies, spot emerging trends on issues or products and track consumer opinion. 'It would be fair to say that most companies don't have their arms around how to gauge the impact that these non-traditional methods have on their brands,' he adds.

Dean Wading, deputy MD of TNS Media Intelligence UK, says: 'You need to pay as much attention to unsolicited and unaided commentary as to solicited opinions. We interview consumers to understand what they are saying in real time about clients, their competitors and their specific brand issues. Loyal customers, who tend to exhibit high levels of CGM activity, can  help clients gain more understanding over the buzz that's generated about a brand.'

Evaluation challenges
Wading and Hampton agree that trackers can be inserted into emails and report back when a message is opened. But techniques to assess viral and WoM effectiveness in brand-building or improving market share and sales have yet to be developed.

Edward Bird, evaluation director of Romeike's consultancy division Delahaye, says: 'The challenge for the evaluation industry is coming up with a method in a language that business understands.'

Judging a viral campaign's success on how much it has been seen – rather than on its ability to shift product – may be of little interest to the people who are paying for it.

Graham Goodkind, founder of Frank PR, says his aim is to tap into ideas that will create 'water-cooler chats'. 'The main problem we have with traditional PR evaluation techniques is they come nowhere near measuring this buzz. Would a shampoo brand manager pay £20,000 to get the brand name mentioned once in a page feature about the UK shampoo market where a dozen other shampoos are mentioned just as favourably?' he asks.

As a result, Frank PR has trademarked the 'Buzz Barometer' and 'Talkability Metrics', which attempt to show how a PR-generated concept has permeated the public consciousness and given rise to chat (see left). I to i research also uses what it calls a 'Buzzometer' to measure how many buzz 'transmitters' (people that have talked about the communication) were active and how many 'receptors' (people who were talked to) were reached in a PR campaign.

Claire Spencer, CEO of i to i research, defines transmitters as people who influence other targets, typically friends, relatives and opinion
formers. 'The receptors are the target audience for the buzz,' she adds.

Interviews with transmitters aim to understand what conversations they have been having about a brand, while research among receptors establishes the channels they have been exposed to. It then interviews buzz aware groups to see if they think or behave differently.

According to Spencer, in the face of media fragmentation, plus web and viral marketing, media evaluation on its own is now too narrow a focus. 'The only way to measure the effect of PR, including viral elements, is to look outwards, starting with how people consume communication,' she says. 'Consumers are being exposed to a plethora of messages and media on a daily basis.'

Practice overtakes terminology
According to Spencer the interviews can be used to change attitudes and behaviour over time. This way, it also claims to be able to isolate advertising, PR, promotions, sponsorship and viral – into constituent parts – using primary research. This includes providing interviews with prompts or visual materials, rather than counting clips or page impressions.

The firm's research was used to track talk around the bid for the 2012 Olympic Games. It showed that while early opinion polls said public support for London hosting the Games stood at 70 per cent, there was a significant group of people (swing supporters) whose support could not necessarily be counted upon and who spread their low opinion of the Games to others. They were, in fact, 29 per cent of the total, with just 40 per cent of London in strong support of the bid.

The research identified a need to shore up swing support and counter the high degree of negative media coverage as more than 20 per cent of Londoners were able to play back a negative message about the bid. Further research in May showed that advertising and PR had helped provide more positive bid messages.
Measuring negative WoM
The company has used similar techniques to evaluate the negative influence of WoM about price rises on British Gas's key performance indicators, such as brand perception and customer acquisition or retention.

But because not everyone is using the same measurement methods, clients are left wondering who is right. Justin Kirby, MD of Digital Media Communications, is also co-founder of the Viral & Buzz Marketing Association and, along with Dr Paul Marsden of the London School of Economics Institute of Social Psychology, points to a way forward.

The pair co-edited new book Connected Marketing and suggest one can measure effectiveness of a WoM campaign in the most telling manner possible.

This involves surveying people before and after a campaign, and asking if they are likely, on a scale of one to ten, to recommend your product. This gives the ratio to which people will
advocate or rubbish  your product: the 'net promotion score' (NPS).

This is where it gets interesting for the client's board; a recent study
by Fred Reicheld at management consultant Bain & Co, published in Harvard Business Review, suggested there was direct correlation between NPS and sales. Marsden says: 'For every five per cent you increase your NPS you boost business growth by one per cent.' His point is that getting consumers talking is not enough. Budweiser's 'Whassup?' campaign in 2000 created buzz – but market share dropped 1.5-2.5 per cent during its run as sales fell.

Marsden adds: 'Most viral marketing campaigns are measured by reach and most buzz campaigns by column inches, but neither correlates to sales.' Millward Brown Precis now plans to integrate NPS into brand audits. 'The best measure is: "Would you recommend this brand?",' says Hampton.

It would be a big step forward for PR if success could be quantified so attractively. It is no surprise that new media channels are appearing more quickly than evaluation techniques can evolve, but there are those who believe that PR is failing to grasp change.

'We've got it slightly wrong regarding evaluation,' muses Goodkind. 'The fact we're debating how to evaluate in an industry that is so much a part of the marketing mix proves that.'

And PROs certainly need to know how to make a client's senior managers feel comfortable. Bird says: 'AVEs have a life in the boardroom because they are understood. CEOs don't know what media content analysis is but they do understand what increasing revenue by £100,000 means.'

Have you registered with us yet?

Register now to enjoy more articles and free email bulletins

Already registered?
Sign in