EVALUATION: Outcome vs output

How do you judge the impact of PR without just counting press cuttings? Adam Hill on systems that claim to do just that.

Receiving a stack of press cuttings about your clients may be a great ego boost. But the reality is these reports - important and useful as they are - often detail the coverage you have gained, but not how your target audience changed its behaviour as a result.

The correlation between business success and positive PR has always been hard to prove. Corporations may justifiably believe that publication of, say, their interim results will move the share price whether there is press coverage or not. One media evaluation practitioner suggests that a lot of evaluation is carried out for its own sake, in a tick-the-box-that's-done fashion.

Proving ROI helps in budget battle

Against this background, Delahaye Medialink Europe research director Steve Virgin spells out what evaluation must be aiming for: 'The Holy Grail is how many bottles of beer did you sell and how you can prove it.'

With PR scrabbling for budget against direct marketing, telesales and advertising, the idea of a return on investment (ROI) figure for PR activity is certainly attractive.

But while the IPR's evaluation toolkit aims to provide some kind of standardisation for the industry, there is suspicion at the 'one size fits all' approach.

'We try to put together an off-the-shelf package, but every client wants something different,' says Xtreme Information (news division) deputy MD Jeremy Thompson. 'It may be to show ROI, be campaign-driven or designed to show the board what the sales team has done one month.'

And while advertising value equivalents have come in for criticism in recent years, many clients still see their value. 'More than 50 per cent of our clients ask for them,' reveals another evaluation company.

Column inches a means to an end

All PR evaluation firms aim to widen the choice available to clients, some as part of broader evaluation services, for anything between £500 and £100,000.

While column inches remain important - as a means to an end - there is at least an attempt to move evaluation away from counting press cuttings - a commodity business - to measuring outcomes rather than outputs.



I to i ('impact' to 'influence') tracker aims to define the links between PR output, its impact, and subsequent influence on attitudes and behaviour. Set up in 2002, it asks consumers what they remember about PR activity, if they support the brand proposition and whether they will do anything about it - that is, buy more products and services.

The methodology starts with a poll carried out by reputation analysis specialist Echo Research. This breaks down a sample of 1,000 people into two groups: 'PR aware' and 'PR non-aware'.

A complex system of values built in to the i to i tracker model then filters the impact of other factors, such as advertising or direct marketing, to isolate the impact of PR. This, the company claims, can enable clients to look at how PR is contributing to a brand's image.

I to i research managing director Claire Spencer says the product will also give PROs a 'cost per influence' figure. 'This is part of an evaluation ladder, for people asking "what can I do to improve my communications?",' she says. 'You may find out there is great frequency and reach, but your messaging is not so good. People have thought about PR being different from advertising, but it's not that different. You just can't control your message.'

It may be effective, but it's not cheap. 'Budget is the biggest hurdle,' Spencer admits. 'If you're spending £250,000 upwards on PR, you can afford the basic level of work with us. It's got to be big companies that drive this.'

Strong, effective messages British Gas director of corporate affairs Patrick Law is pleased with results. 'We started working with i to i in 2002 and have undertaken three waves of research so far,' he says. 'We're trying to search out information that helps us direct comms efforts to help the business. We're trying to determine that our messages are sufficiently strong enough to influence the consumer. You're not measuring the number of articles or opportunities to see, you're measuring what people's preferences are and seeing whether they have been influenced.'

Supplier: i to i research/Echo Research

evaluation extra)

Office of Information



Xtreme Information's Impact Check was launched in December 2003, and is a clear extension of the company's former incarnation as an overnight press clipping agency. It aims to provide clients with a swift means of establishing the effect of media coverage. A company can see a news story about itself, a competitor or client at 7am, request the service by 10am and receive a preliminary report on its effect on the public by 5pm. Full results are delivered the following morning by email (Monday for Friday polls).

A poll of 1,000 adults, conducted by YouGov, asks two questions on behalf of the client. These are, essentially: did you see the story? And what did it make you think of the company? In theory, this will reveal who saw the story, their reaction, and how their perceptions of the company, brand or individual have been affected. The results are tabulated by age, gender and social class, and have five regional splits. Responses to the second question are broken down by those who saw the media reports and those who didn't.

'Traditional media evaluation has focused on monthly reports that are very interesting, but don't allow you to react,' says Xtreme Information (news division) deputy managing director Jeremy Thompson. 'Impact Check strives to offer some input into outcome as well as output. If the story is related to a specific campaign, three months down the line isn't good enough.' At £500 for two questions on one topic it is relatively inexpensive, although Thompson stresses that clients are expected to use it only ad hoc. 'It is still new and is not an ongoing subscription service. You wouldn't use it every week or month. Impact Check is unique right now, although before too long I am sure it won't be,' he admits.

National Rail Enquiries chief executive Chris Scoggins used Impact Check to evaluate a recent PR campaign and admits the programme worked well: 'To get two responses in a day coming back automatically was fantastic. It has certainly given us an insight into the effectiveness of our communications.'

Supplier: Xtreme Information/YouGov

Cost: £500 for two questions on an issue

Clients: National Rail Enquiries, Procter & Gamble, Mitsubishi Motors



ATP Consumer Tracking from Millward Brown Precis is designed to be used in conjunction with the company's wider evaluation service, the Media Influence Index on key messages, which gives each piece of client coverage a score out of 100.

'We can gauge the impact of PR on consumers at many levels,' claims Millward Brown Precis managing director Fergus Hampton. 'For example, does it help drive spontaneous awareness, familiarity or the propensity to buy? Through balancing out all the input, we can judge the genuine contribution of PR.'

ATP compares the responses of consumers who have seen the relevant media item with those who have not. 'People get confused,' Hampton claims. 'If you're close to the brand, you're more likely to say "yes, I've seen PR" regardless of what has actually happened.'

The company says ATP can break down whether, say, PR connects best with consumers when communicating specific messages or general ones, such as dependability, about the brand. 'You get the best cut-through in the automotive sector,' Hampton adds.

The product can also be used to show how PR contributes to consumer opinions about a product at different stages of its life cycle, from launch to maturity.

Quick, easy guide

Volvo uses ATM consumer tracking to track its PR progress, as well as that of its competitors. Product affairs manager John Rawlings says: 'We used to just measure how much coverage we got each month. ATM gives us a very quick, easy guide. The programme is not cheap, and it takes up a relatively large part of our budget. But we have been using it since August 2001 and are very content with the system.'

Supplier: Millward Brown Precis

Cost: £25,000-£100,000+ per annum depending on whether study

is continuous or pre- and post-campaign, and extent of analysis

Clients: Peugeot, Ford, Coca-Cola, Barclays Bank, Unilever, Volvo

Have you registered with us yet?

Register now to enjoy more articles and free email bulletins

Already registered?
Sign in