THE PROOF SURVEY - PR agencies reveal the truth behind their R&E practices

It was just over a year ago that PR Week first launched the Proof campaign in a bid to encourage client companies to allocate 10 per cent of their budgets to research and evaluation (R&E). The campaign has evolved considerably in the last 12 months, with the focus moving away from setting target spending figures to examining the reasons for the reluctance to spend.

It was just over a year ago that PR Week first launched the Proof

campaign in a bid to encourage client companies to allocate 10 per cent

of their budgets to research and evaluation (R&E). The campaign has

evolved considerably in the last 12 months, with the focus moving away

from setting target spending figures to examining the reasons for the

reluctance to spend.

A series of straw polls undertaken by PR Week over the last year

highlighted the need for greater emphasis on educating client companies

and PR practitioners about the benefits of proper planning and

evaluating PR activities. At the Proof Forum last November a major step

forward was taken when the IPR and the PRCA committed to co-funding a

set of best practice guidelines on the use of R&E. These guidelines,

which will take the form of a toolkit, are currently being written by

Michael Fairchild, author of ICO’s publication ’How to get real value

from public relations’ and overseen by an editorial board chaired by PR


As part of this process, PR Week has commissioned the most in-depth

survey ever undertaken into the use of R&E by the UK PR industry. The PR

Week/ Countrywide Porter Novelli Proof Survey was carried out by CARMA

International and involved telephone interviews with over 200 PR Week

readers and a series of face-to-face interviews representing a

cross-section of both in-house and consultancy roles and disciplines.

This report is also the first of its kind to compare the views of

agency, in-house and public sector practitioners.

Time for action

The Proof Survey makes for disturbing reading and shows that a year on

there is still much to be achieved. Although most people within the

industry are keen to jump on the bandwagon and state their view that

better R&E is crucial to the success of PR, the evidence is that many

are simply paying the issue lip-service. On the one hand, the Proof

Survey shows that PR practitioners feel the main challenge facing the PR

industry in the UK today is improving its image. On the other, people do

not appear to associate this with greater accountability and acceptable

measurement standards. ’This seems to be a fundamental mis-match,’ says

Christine Woodcock, client services director at Countrywide Porter


It was because of the need to move the debate beyond politically-correct

affirmations of the importance of evaluation, that PR Week decided to

try to establish a more detailed picture, not only of how practitioners

working in different PR disciplines view, but also how they use, the

different R&E methodologies available to them.

Eager to practice what they preach, PR Week, the IPR and the PRCA also

wanted to establish a benchmark against which the success of the

forthcoming toolkit can be measured in terms of its effectiveness in

raising awareness of and increasing use of different R&E methodologies.

By looking at the various existing attitudes and applications of R&E, PR

Week and Proof Forum members plan to draw up a framework of measurable

objectives to encourage a wide audience to rethink their planning and

evaluation techniques.

This looks likely to be a tough task. A total of 20 per cent of

respondents said that the success of their efforts could not be

evaluated; this included 23 per cent of internal communications

specialists, 25 per cent of those working in community relations, 28 per

cent of those working in financial PR and 67 per cent of those working

in government relations.

As part of the Proof Survey, respondents were also asked to talk in

detail about which methods they believe can be used to plan and measure

their activities, and which of these they have actually used. These

responses included some significant disparities and omissions.

Not surprisingly, the most commonly used method across all disciplines

was media content analysis and press cuttings, closely followed by media

reach/OTS. It is interesting to note that 61 per cent of all respondents

said they have used the technique as a means of planning and evaluating

their campaigns, while just 34 per cent regard it as an effective tool

for convincing budget holders to part with their cash. This includes,

remarkably, more than three-quarters of the internal communications

specialists interviewed. Media content analysis and press cuttings was

quoted by 77 per cent of internal comms specialists as their favoured

method of evaluating their work, while only 38 per cent considered this

a suitable method for communicating their effectiveness to PR

budget-holders and decision makers.

However, long-term industry pressure to discourage use of Advertising

Value Equivalents (AVEs) seems to be bearing fruit. In comparison with a

PRCA survey in 1997 that revealed use of AVEs among 63 per cent of

practitioners, just nine per cent of respondents to the Proof Survey

named the technique as a measure they use.

Once you stray beyond the traditional boundaries of media analysis,

however, there is a significant falling off.

Less than one-third of all respondents spontaneously mentioned surveys

or polling as an R&E tool and alarmingly, less than one-third of

consumer PR practitioners named consumer surveys as a possible measure.

In the community relations field, for example, 44 per cent of

practitioners saying that they should be using consumer surveys but only

25 per cent actually do - one-third of respondents said they rely on

’gut feel’ and anecdotal information to judge the success of their


In addition, techniques such as focus groups were not widely popular -

only 22 per cent of consumer PR people quoted focus groups as a possible

tool for planning, with only 15 per cent actually making use of


Only five per cent of corporate PR practitioners viewed share or sales

price increases as an indicator of PR success. And, despite the efforts

of the Government to promote dialogue between local authorities and the

communities they serve, only 16 per cent of public sector respondents

said they believed interviews with target audiences to be useful for

planning communications.

’The survey highlights one of the greatest weaknesses in existing

practice - only three per cent have ever used pre-testing, a methodology

which the toolkit argues strongly for,’ points out Fairchild. ’How can

you develop messages for target audiences without knowing whether they

can be understood and actually work?’ he asks.

One of the greatest stumbling blocks to a more wide-ranging approach to

R&E appears to be that the perceived links between PR activity and the

bottom line need to be improved. While 91 per cent of respondents agreed

that a company’s reputation is a measurable asset, almost half felt that

PR activity is rarely tied into business objectives.

In addition, there is a direct contradiction between what people say are

the main advantages of R&E and what they do. All sectors stated that

showing the value of their activities and demonstrating how far

objectives have been met are key benefits of R&E. Yet when pushed, they

seemed unable to back this up with solid examples.

The future of evaluation

To ascertain how the PR industry feels about a set of common measurement

standards, the Proof Survey asked respondents what they think the main

obstacles to planning and evaluation to be. Surprisingly, around

one-third of business-to-business and consumer specialists cited lack of

definable measures and 23 per cent of all respondents pointed to lack of

time. However, the main complaint was the difficulty of obtaining

budgetary funding.

To examine this further, respondents were quizzed about selling

evaluation to clients, directors and sponsors. Almost half the

respondents actively said it was a problem. However, more than

three-quarters felt a set of common standards would make selling

measurement easier and more than 80 per cent welcomed the concept of an

evaluation toolkit.

But just what is the long-term commitment of the PR industry to planning

and evaluation? Are practitioners really going to make the best use of

the IPR and PRCA toolkit? ’I would welcome any kind of manual, but it

needs to be backed up by some kind of workshop,’ says one


Certainly among Government departments, this sort of talk has to be


When asked whether the PR industry needs to improve its evaluation

efforts, 50 per cent of public sector workers strongly agreed. Yet, in

response to their personal commitment to measuring activities, this

strength of feeling dropped to under one-third.

On a brighter note, in-house and agency staff were more positive and 85

per cent of overall respondents agreed they were personally committed to

evaluating their own efforts.

The final truth will come out in the real allocation of funds.

Significantly, a question designed to elicit the budget spend of

respondents’ most recent projects produced a high level of non-response.

Stephen Welch, head of market research at CARMA International says it is

not unusual to meet this level of resistance to probing about finances.

However, he is surprised by the number who shied away from revealing

what proportion of this budget was spent on evaluation. He thinks this

may be attributable in part to people who prefer to give no answer,

rather than a perceived negative.

Of those companies who did respond, the average spend was seven per cent

of total budget, although for most projects, this spend in real terms

amounted to little more than pounds 1,000.

CPN’s Woodcock says: ’It is painfully telling that 43 per cent of

respondents either refused or gave no answer when asked their evaluation


It rather implies they realise the wisdom themselves, but still need

help - and proof - to convince senior colleagues of the benefits of


Have you registered with us yet?

Register now to enjoy more articles and free email bulletins

Already registered?
Sign in