PUBLIC RELATIONS: How to measure the effectiveness of PR - The PR industry believes it has created tools to quantify success. Mary Cowlett reports

Persuading those who hold the corporate purse strings that PR is a cost-effective exercise with measurable results has always been a problem.

Traditional evaluation techniques, such as weighing media cuttings, have not produced the hard evidence the doubters need.

But now the PR industry has made a dramatic breakthrough. Over the past eight months, there has been a rash of new PR measurement tools, whose creators claim can prove and quantify success, measuring outcomes rather than output.

At the end of September, Manning Selvage & Lee launched its I to I tracker which, it says, goes beyond the conventional gauges PR has employed, to meet more meaningful measures.

Already, this tool has evaluated the PR value received by Western Union from its sponsorship of this year's Notting Hill Carnival.

'We've applied techniques clients are used to seeing from other marcoms agencies, like reach and frequency figures,' says account planner Pamela Neira. 'This means we can evaluate shifts in attitudes and behaviour.'

It also means that in conjunction with Research International, MS&L seems to have cracked the old chestnut of disaggregation, or rather, differentiating PR initiatives from other marketing activity among the general public.

According to the MS&L team, this has been achieved by developing a panel of respondents who are both brand aware and PR savvy.

Measurement kit

In a similar vein, Hill & Knowlton recently teamed up with WPP sister firms Millward Brown and MindShare ATG to launch a measurement kit, called PRecision. This has three main components: media impact measurement, consumer or stakeholder attitude tracking and strategic media targeting. 'We believe that PR should be accountable in the same way as any other marketing tool,' says Mary Lee Sachs, H&K executive vice-president of worldwide marketing communications.

'The PRecision range of tools monitors not only how many people are receiving PR messages, but also the quality of these messages, their prominence and the duration and size of exposure. When married with traditional tracking methods and strategic planning, that means we can isolate the PR effect.'

While of enormous benefit to clients, these new tools also have a beneficial impact on the PR industry as a whole, enabling the communication professionals to stand shoulder to shoulder with other marketing specialists. They could also be a valuable weapon in fighting off the current threat to PR from internet specialists.

According to the PR industry's professional bodies - the Institute of Public Relations (IPR) and the Public Relations Consultants Association (PRCA) - there has long been a need for better evaluation techniques - if only to provide demonstrable evidence that PR really works.

To this end, in February 1998, the industry's key trade bodies, the media evaluation and research industries, came together to create the Proof Campaign. The aim was to push research and evaluation up the agenda to create greater accountability and solid professional standards.

Real-life examples

Last summer, this led to The Public Relations Research and Evaluation Toolkit funded by the IPR and PRCA. Written by PR consultant Michael Fairchild, this kit is a guide to the use of research and evaluation, packed with real-life examples from organisations including the British Heart Foundation.

But the guide has not taken off as its endorsers hoped. 'Our recommendation that 10% of the PR budget should go on evaluation has served its purpose in highlighting the case for research and evaluation, but now it may be working against us,' says Fairchild. 'It has made it too easy for the client to separate research and evaluation from the project. Clients now think they can save 10% of their budget by discarding evaluation.'

This autumn, in a bid to rectify the situation, the PR industry launched its ultimate weapon, a PR campaign. Run by industry taskforce, PRE-fix (an acronym for Planning, Research and Evaluation), this latest initiative aims to bang the drum for PR measurement to client organisations.

Martin Loat, PRE-fix chairman and managing director of the Propeller Group, says: 'We are looking to preach to the unconverted and create a downward pull for greater PR measurement from the pay masters on the client side.'

This approach is particularly pertinent in the wake of the Company Law Review. Pressure on company directors to take account of stakeholders' interests, and to provide improved social and environmental reporting could, in effect, mean companies have to put a price on their reputations.

'New legislation may mean that all the board directors of an organisation will be interested in what effect their investment in PR has on the company balance sheet,' says Loat.

The PRE-fix campaign is rolling out with a conference and contact programme aimed at placing speakers at key business events, such as CBI gatherings.

In the new year this will be accompanied by a quick and easy guide to planning and evaluation, backed by a PRE-fix web site. 'The benefit of the campaign to PR practitioners will be a better understanding about the importance of evaluation among clients,' says Loat.

One of the historical problems for measuring PR is its complexity, the diversity of audiences and the way that PR interacts with an organisation's other disciplines. This means there is no simple right or wrong way for undertaking evaluation.

Flawed techniques?

That said, there are a few techniques - still in current usage - that leave top PR practitioners dumb-struck with horror. The most notable of these is Advertising Value Equivalents, or AVEs - where editorial coverage is measured in inches - or in the case of broadcast coverage, seconds - and given a monetary value equal to a similarly placed ad. 'AVEs are flawed by the fact that advertising and PR use quite different methodologies,' says Fairchild. 'Valid comparison is therefore difficult, if not impossible.'

Another popular ploy is Opportunities To See (OTS), which in its favour, provides a useful quick guide. On the downside, it does not quantify people's so-called out-take - what the target audience has taken away from the PR activity.

But, according to Richard Merrin, director of Spreckley Partners, companies still demand these types of evaluation techniques. He says his agency works with one IT client that requests an annual AVE-based report and another client, Avery Office Products, that favours OTS measures. But in the case of the latter, Merrin says: 'OTS is only part of an overall evaluation package, focused on delivering solid business objectives.'

But by far the most popular measurement technique used by the PR industry is media analysis. According to a recent study by Echo Research and industry magazine PR Week, 88% of in-house PR departments evaluate campaigns this way - rigorously scouring press cuttings to evaluate if messages have been communicated.

This situation leaves many frustrated; suspecting that far too many PR practitioners equate their trade solely with media relations. 'Monitoring the results of PR campaigns is clearly important, but there is a lot of nonsense talked at the same time, especially about the value of spending lots of money analysing press coverage,' says Tim Prizeman, director of Kelso PR. He is concerned that evaluation by media coverage alone ignores whole swathes of company stakeholders, including internal audiences, suppliers, analysts and, of course, investors.

But a far greater worry is that in isolation, media analysis tends to tell only half the story, focusing on PR output, rather than how well a PR campaign has had an impact on and influenced the target audience.

Media analysis companies are keen to address this criticism. 'It's well recognised that media monitoring and analysis are sophisticated tools,' says Mark Westaby, chairman of AMEC (Association of Media Evaluation Companies) and managing director of Metrica. 'The question now is: 'We understand the output from the media, but what's caused the change among the target audience, is it the sales promotion, the advertising, or the editorial coverage?' This is something the industry is looking into seriously.'

Already, many traditional media measurement companies provide a more holistic approach to evaluation. Surrey-based Echo Research has been offering clients an integrated research and evaluation package for More than two years.

'We can help people identify not only what's being said in the media, but also customers' attitudes, their intent to purchase, their behaviour and the financial performance of an organisation,' says Echo's market research director, Stephen Welch.

This is achieved by media analysis to measure output - column inches and favourability ratings - and opinion research to examine out-take.

But Welsh says he can also provide a financial value for communications exercises. 'For a client in the mining sector, we showed a close correlation between their media coverage and their share price,' he says.

But what worries Westaby is that such solutions tend to be bespoke and favour the big guys with the big budgets. 'If you are a global brand, then it's probably worth spending hundreds of thousands of pounds to find out what effect your PR is having, but that leaves out the other 99% of organisations,' he says.

Great expectations

So what do the people who really matter, the in-house communications and marketing practitioners, expect from their PR evaluation? Do they want to be able to measure PR in the same way that they measure advertising?

'Yes, if that were possible,' says Matt Peacock, director of corporate communications at AOL. 'But PR is a subtle mix of reactions to a brand, by diverse audiences, so it's not as simple as 'have you seen these ads?'.'

As an ISP, a key element in AOL's corporate strategy is how the company and its brand are perceived by its audiences. 'When we launched our Flat Rate Service in September, the important benchmarks were the extent to which journalists understood what we were offering and the degree to which they supported it,' says Peacock. But he is not convinced that anyone can put a truly quantifiable value on each PR pound spent.

This is a debate that will run and run, but it seems certain that more rigorous PR evaluation is here to stay. The only concern is that measurement becomes an integral part of the PR process, rather than a bolt-on justification for what has been achieved.

'There is huge pressure now to show PR value, and some still try to do that retrospectively, which is short-sighted,' says Crispin Manners, chief executive of Kaizo PR. 'At the outset both parties should plan what success will look like and then deliver it. That way, it forces as much discipline on the client as the agency and you both know what you want to achieve.'

REACHING THE TARGET AUDIENCE

This year, Manning Selvage & Lee worked with the Health Education Authority to instigate a consumer education campaign on food hygiene.

From the outset, the PR team set themselves measurable objectives, which by the end of the campaign proved they had achieved a real change in attitudes and behaviour.

The target audience was primary school children. This was supported by findings from BMRB, which showed 67% of children regularly prepared food at home, but 45% of youngsters did not wash their hands when handling food.

It was apparent that the most cost effective and authoritative way to reach this group, was to speak to them in school, using their classroom teachers as key influencers.

To capture the imagination of the children, a schools pack was developed using cartoon characters 'Safe-T and the H-Squad'. Packed with exercises, games and advice on how food hygiene could be integrated into the lesson plans, these kits were distributed to every primary school in England and Wales.

To support this activity, the PR team implemented a media campaign, targeting both children and their parents. Cartoons were placed in the Beano and the Dandy, while research on children's attitudes to food hygiene was used to create interesting new angles for adults. This was boosted by a launch event in London, featuring celebrity chef Lesley Waters.

In June, a media analysis report from Metrica, showed that the media campaign reached two out of five mothers and over half of all primary schoolchildren at least twice. In addition, at least one key message about food hygiene appeared in all coverage.

But comparing data from BMRB's Children Access Survey findings in October 1999 with May 2000 provided the most interesting results. This showed that 53% of all 7-to-11 year olds were able to recall the PR activity on food safety and 51% covered food safety in class.

In addition, the survey revealed that awareness of getting food poisoning from poor food hygiene practice had risen from 36% pre-campaign to 72% afterwards. Awareness of the importance of hand washing when handling food had also risen, from 60% to 75%. 'The evaluation for this campaign will help educate the planning process for future health education campaigns - ensuring they reach and affect the target audience in a meaningful way,' says MS&L account manager Claire Martin.

BEST PRACTICE IN PR EVALUATION

- Get all interested parties to buy into a comprehensive brief. This helps to avoid common problems such as failure to identify the real objectives and purpose of a PR campaign or project, poor definition of target audiences and objectives that

are unrealistic or too numerous.

- Set clear objectives. PR objectives must reflect the strategic and business goals of the 'client' organisation. For the chosen PR objectives to be measurable they should identify target audiences, key messages to be delivered to precise audiences, the medium, the desired response and timing.

- Decide on your own definition of success, then select the most appropriate method of evaluation: output records what messages went out, such as media coverage achieved, and the audience reach. Out-take measures to what degree PR activity changed the audience's opinions, behaviour or attitudes. This is the most valuable measure of all as it allows you to plan your future campaigns more accurately.

- Use research throughout the process - before, during and after. Research plays a role throughout the process, helping to identify issues up front, as well as tracking the progress of the campaign and its outcome.

- Pre-testing of messages and tactics to see how they work in practice with target audiences is also desirable.

Source: PRE-fix.

This article was first published on Marketing