If the Institute of Public Relations has its way, discussions about
the role of PR will no longer be framed by warm words and woolly
Instead, the new language of the industry will be about graphs, facts
and figures. All of which will help prove the effectiveness of the
industry to clients.
The first update of the IPR's Planning Research and Evaluation Toolkit
promises to help the industry elbow aside competitors from other
marketing disciplines to take its place at the client's top table.
'What it really does is elevate the role of PR. For practitioners who
use the toolkit and carry out proper research and evaluation, it can
demonstrate the strategic importance of PR,' says The Fairchild
Consultancy MD and toolkit author, Mike Fairchild.
IPR chairman Ian Wright says the second toolkit aims to build on the
progress made by its predecessor: 'Practicioners were starting to
evaluate and recognise the importance of measurement, but they were all
doing it in different ways. The toolkit provided a standard approach
that could be used on any campaign.'
And the IPR says interest in the issue has been strong with nearly 1,000
copies already sold, on the back of wide recognition that evaluation is
an important issue for the industry.
Wright adds that despite occasional difficulties with evaluation, the
industry must continue to work hard to improve its position: 'All the
FTSE 100 have PR departments as they see the value of reputation
management, even though it is not straight measuring an intangible.
However, to progress and go further we must be able to measure and
evaluate what we do more thoroughly otherwise we will be seen as the
poor relation in the marketing mix.'
Fairchild says the new edition has been substantially updated from the
April 1999 version, with an increased number of case studies.
Among the changes are: details of how to combine new research techniques
to produce more accurate results; ideas for identifying the separate
effects of advertising and PR; techniques to help analyse post-campaign
research to assess the impact of PR on the people who have been exposed
to it; guidance on how to adapt planning, research and evaluation
methodologies to the internet; suggestions of ways to measure the value
of keeping a client out of the media; and ways of translating PR
effectiveness into a return on investment figure for the client.
Fairchild also argues that the industry needs to move away from using
Advertising Value Equivalents, which he says are an unreliable tool for
measurement. AVEs, he says, compare PR work to advertising when in
reality performance should be measured against the PR objectives that
have been set.
However, some questions are being asked about the new toolkit. Professor
Jon White, who teaches PR at City University, says that the very term
planning, research and evaluation is wrong.
'The PR process involves a number of steps - first of all research, then
planning, then an action phase, and then evaluation,' he says. 'For some
reason the toolkit is taking the whole established process and
suggesting that planning precedes research.'
White also questions who the guide is aimed at. 'The toolkit encourages
people to follow a process and my point would be that process has got to
be ingrained right from the outset,' he says. 'It's not something you
can come to after ten years of practice. It's got to be ingrained.
'An effective PR practitioner should be doing what the toolkit is
telling them already so it wouldn't really help such a practitioner. If
they need it they can't have been very effective.'
He adds that while the toolkit is a useful attempt to capture current
thinking, the industry needs to address the wider issue of training for
people coming into the business: 'The weakness across the board in
evaluation is to do with the way that practitioners are prepared for
work in practice. We are really looking at questions of education,
training, the early work in practice. These questions are not going to
be addressed by a thin document.'
RAC Foundation executive director Edmund King found the toolkit
reassuring as the independent motorists group already uses most of the
suggested techniques, although not necessarily all in one campaign.
'I agree with most of the theory. Where I think it has gaps is when it
comes to the reality on the streets. In any ongoing campaign using wit,
speed and opportunism is often as important as planning, research and
evaluation,' he says.
It's a similar case, he argues, when the toolkit suggests that planning,
research, evaluation 'does not have to be funded out of the PR budget.
As PR reinforces corporate reputation and provides benefits beyond the
remit of PR'. He says: 'In the ideal world it might be funded out of
other budgets but often it is not.'
King applied the toolkit to the RAC's recent 'Don't Dig There' campaign.
The campaign focused on the problem of utility companies digging up the
roads causing excessive congestion and costing the UK economy £2bn
per year. The aim was to get the Government to introduce legislation
that penalised companies if their work was not completed on time. The
RAC Foundation linked up with the Evening Standard to highlight daily
stories of bad practice, briefed leader writers of The Times to write an
editorial and convinced the BBC's Watchdog programme to cover the
'Despite the fact that the RAC Foundation uses ongoing media evaluation,
the use of media in this campaign was not evaluated specifically as the
objective of the campaign - to get legislation introduced - was won,'
Perhaps the most reassuring thing about using the toolkit was that
looking back on decisions made to short deadlines, the bulk of its
techniques had been followed as a matter of course: 'Sometimes in the
hectic life of a high-profile campaigner instinct has to take over. One
does not always have the time to check out the six steps before reacting
and embarking on a newspaper campaign. But when looking back, we have
normally adhered to the toolkit principles without consciously realising
it.' Overall, he says, the toolkit is a good start but the campaigners
should remember to be on the lookout to grasp 'opportunities with their
bare hands, to sometimes take risks and use creativity to get the story
across. However, as a tool to remind us all of best practice, this
toolkit is as good as they come'.
AMANDA HARRISON - MEDIA
For Amanda Harrison the reality of life as a sole trader means that
applying the toolkit's suggested research and evaluation techniques to
her work gets short shrift.
'I felt it was unrealistic as a sole trader. If I followed the toolkit
to the letter, I would spend all my time proving my PR rather than
generating PR for my clients,' she says.
She concedes that the techniques will work better at a large agency
where account spends of upwards of £10,000 a month might justify
the time required.
'In a climate where you have an account team of three working on the
whole process, there's the time to apply post-campaign analysis,' she
In fact, she argues, the reason why clients opt for a sole trader is
that they much more personalised service and as a result they judge the
effectiveness of her work very quickly.
Harrison specialises in the marketing sector. Her clients includes
Quantum Media - an online and interactive media agency - London Taxi
Promotions, Continental Research and advertising agency WFCA
One of the challenges of applying evaluation in this kind of work is
that as agencies tend to be promoted as a brand on the basis of the work
that they do. Setting simple objectives when much of the activity
relates to the client's brand becomes 'way, way too complicated'.
'Clients' auditing of my work is down to the coverage I generate as well
as managing the process in a downturn of news,' she says.
Clients of Amanda Harrison Media also receive text messages when big
news stories break such as William Hague's resignation after the
election and the merger of Compaq and Hewlett-Packard.
'It's very difficult to audit that. Have you given me enough news?' she
A former journalist, Harrison has been in PR since 1997, and has worked
at MacLaurin and The Media Foundry. She says that the toolkit will be a
'Just as a way reassuring yourself that you are on the right track in
moments of darkness,' she says.
However, she does have other reservations.
The presence of a major grammatical error in the introduction was
disappointing. 'Full marks for wanting some kind of proof that the
industry is full of capable professionals but after that I took the
whole document less seriously,' she says.
And the reality is that even in larger PR organisations, evaluation is
often the last item on the 'to do' list. 'Auditing, from my experience,
seems to happen when you are trying to retain a client, by which time
you've probably lost them,' she says.
Age Concern campaigns manager Scott Davidson applied the toolkit to the
charity's recent health campaign.
The aim was to highlight the fact that ageism was a real issue in the
health service and that it was endemic rather than being a few isolated
His main impression was that while the toolkit has all the right
process, there is a real question mark for smaller organisations as to
how much time they can spend on research: 'Small organisations have to
think very hard about how they dedicate their research, sometimes you
have got to get out there.'
For the health campaign, Age Concern needed to look into the legislation
and guidelines covering the area, working out who was responsible for
the situation and how it might be changed. 'The key questions are: how
do you get there, does it need legislation or new guidance and who's got
the power to make the change that we want?' he says.
The crux of the issue was that while no-one disagreed with the aim of
providing a non-ageist health service, they needed to be convinced that
it was a real problem.
This was done by providing real people to talk to the media, cases where
medical notes proved that staff had unilaterally taken action based on a
patient's age. The level of coverage and the kind of message being
conveyed was monitored by media evaluation officer Margaret
Davidson says the campaign hit a nerve and rapidly produced the desired
result. Having launched the drive in summer 2000, the Government is now
actively involved in dealing with the issue.
In the spring it announced a national service framework and a full
programme to deal with the problem. Age Concern is now working with the
Government and testing local monitoring to see if the new measures have
made a difference.
He argues that with a primarily public affairs task, assessing the
effectiveness of the work is fairly straightforward: 'With public
affairs, we know what we want and either we have got it or we need to
convince someone else.'
One thing that might improve Age Concern's systems might be to
incorporate third parties such as health professionals to guage how they
have responded to the campaign. Davidson says the toolkit will be most
useful at the certain points of a campaign, such as the development
stage or in a mid-term assessment.
Two things that might have made it better were more detailed case
studies and some kind of guidance as to which of the research techniques
are essential: 'It needs some attempt to say here's the core essential,
where you are not going to have a clue where you are unless you do these
things and this is what you should do if you want to be really
Of PRWeek's four 'guinea pigs', QBO director Jo Carr was probably the
most familiar with the toolkit and some of the thinking from the first
edition is already part of the agency's everyday routine.
'I think that it's good in that it underlines that the whole industry is
taking evaluation really seriously,' she says of the new version. The
real challenge in implementing evaluation is to ensure that there are
targets in the first place. 'The big thing for us is getting clients to
set measurable objectives in the first place,' she says.
QBO, which has a client list that includes Royal Mail, Smile and British
Gas, road-tested the toolkit on a campaign for Norwich Union Healthcare
aimed at first-time dads.
The premise was that having children is often a trigger point to
consider health insurance. It looked to build a relationship with
first-time dads as most information is aimed at mums.
QBO tested the idea with focus groups of recent fathers to check that it
was right in thinking that dads could feel overlooked, there was an
information gap and that Norwich Union could be a credible
This pre-testing allowed QBO to tighten objectives and finalise
messages. At this stage it also sat down with the client to discuss how
the campaign might be measured.
As well as ongoing measurement, the media 'output' was also analysed at
the end of the campaign. A further measuring came from the response to a
hotline number. 'This is a good way of checking "out-take" - that the
audience has received and understood the message,' she says.
However, Carr argues that extending evaluation to the level where you
can demonstrate ROI in many cases depends on the client: 'Evidence is
really hard to get and it's sort of out of our control in many ways.
It's about the marketing team talking to the sales team talking to the
client services team.'
While it's something that happens with Smile - where Carr reports that
PR acquisition costs compare 'incredibly favourably' with other forms of
marketing - 'I think it requires the client to have given it thought
internally as do how you get that information.'
Another issue is the toolkit's attitude to Advertising Value Equivalent
figures. 'The toolkit is very negative about AVE and for some clients
that's what they are familiar with,' she says. 'You need to talk in some
terms that are recognised by clients.'