Public Relations: How do evaluation techniques work? - New evaluation methods have an essential role to play in future PR campaigns, says Stephanie France

As clients demand greater accountability, PR practitioners are

focusing on how best to evaluate their campaigns. But for those new to

evaluation, the range of methodologies on offer can be confusing. Media

content analysis and press cuttings is the most commonly used technique

but a range of acronyms such as AVE (Advertising Value Equivalents) and

OTS (Opportunities to See) also promise benchmarks against which

agencies can evaluate their efforts.

The Association of Media Evaluation Companies has a web site

(www.amec.org.uk) which attempts to bring some clarity to media

monitoring but, in an industry which is increasingly placing evaluation

at the heart of the client promise, keeping abreast of new techniques is

not easy.

Looking to the latest innovations, Carma International says its ’Reach

and Frequency’ methodology helps agencies by providing figures that can

be equated to advertising research. ’In addition to measuring volume and

circulation figures, we use national readership data to calculate reach

and frequency, and occasionally gross rating points,’ explains Peter

Christopherson, Carma’s business development director.

’If you book an ad campaign through a media planning agency, you may aim

to reach 80% of your target audience on five occasions - the cost will

depend on size of ad, edition and placement. By using the same media

data to ’post-rationalise’ a PR campaign, you can calculate ’Reach and

Frequency’ and link this to the cost of the campaign, which is more

effective than calculating the cost of having ’bought’ the space in the

first place.’

Comprehensive approach

Both Christopherson and Mark Westaby, managing director of media

evaluation agency Metrica, warn PR agencies against evaluating their own

campaigns.

’PR agencies keep coming up with their own systems because their clients

demand evaluation. They don’t have access to the same audience figures

and readership studies as us. They may say, ’we got great coverage’, but

is it getting through to the target audience?’ asks Westaby.

Media evaluation agency Mantra aimed to offer a more comprehensive

service when it launched Mantra Quick last year. This allows PR

practitioners to evaluate their campaigns for a one-off fee. Partner

Brian Moore says there is no limit to the number of clients it can

serve, but does advise users, which include The Red Consultancy, Firefly

and Cambridge County Council, to take more in-depth analysis to a

specialist. ’Mantra Quick is our own software,’ explains Moore. ’We sell

it as a full service, with training, updating, a telephone helpline and

advice on how to get the best use out of it.’

In the media monitoring field, Durrants Press Cuttings also introduced a

new software system last December which automatically faxes clients

notification of their broadcast mentions.

Normally, media monitoring companies telephone clients with details of

the broadcast tapes they are offering. Sales manager Richard Stobart

claims that Durrants is cheaper and quicker than its competitors and the

service takes less time out of a busy PR practitioner’s day.

’We are following the European trend,’ he says. ’Our software systems

compile the reports automatically, so we don’t have the overheads of a

massive sales team.’ Durrants’ overseas department deals with 96

countries so agencies can be confident that monitoring on international

campaigns is comprehensive.

Tools for planning

Over the past year, PR practitioners have found that their clients

expect more from the evaluation service they provide. This, explains

Stobart, includes ’filtering’ media coverage according to its value.

’There is so much media around these days that the big clients don’t

want every last mention of themselves, such as passing references.’

Planning is another important service that PR practitioners are being

asked to perform more frequently.

Claire Spencer, planning director of Manning Selvage & Lee, says: ’The

planning process must be fully integrated to evaluate a campaign

properly. Unless you have a clear definition of the target audience from

the outset - both demographic and pyschographic - no evaluation, however

good, will be able to prove the effect of your activities on that

audience.’

Westaby agrees: ’The media analysis we do is no good if you can’t relate

it back to the business objectives that the client is trying to

achieve.’

Metrica has just launched a new planning tool called Comm-Map

(Communication Measurement, Management and Planning). ’The system helps

you plan, weight and prioritise your objectives and then adapt your

programme effectively,’ explains Westaby. The tool has already been used

by Tandem, now part of Compaq, to help define and refine its goals and

objectives.

The need for more robust evaluation is showing every sign of gaining

momentum. PR agencies which wait for clients to request evaluation

before acting, may find they are sidelined in the future.

MONITORING MINDSTORMS’ IMPACT

Company

Lego

PR Agency

Manning Selvage & Lee

Budget

PR campaign: pounds 15,000

Evaluation

pounds 1750 (additional)

Background

In January 1998, Lego Group appointed Manning Selvage & Lee (MS&L) to

build a campaign around the launch of Lego Mindstorms. The product was

to receive a big push in October 1998, in time for Christmas.

The campaign, which was not supported by advertising, relied on planning

and research to identify its key users and strategy. The key was to gain

’Big Brother’s’ endorsement, to help position the Mindstorms set as

’cool’ and technologically advanced in the eyes of the target audience

of 12- to 14-year-olds.

MS&L was charged with evaluating the success of the campaign in the UK

market. It worked with CIA Medianetwork to evaluate the reach of the

target audience, Metrica to evaluate the success of message delivery,

NOP to track awareness and propensity to buy, and Carrick James

Childtrack to measure the percentage of product awareness, using a base

of 808 seven- to 14-year-olds.

The Campaign

Prior to the October push, MS&L conducted a media tour of men’s and PC

magazines, including T3 Magazine and PC Format, using a prototype of the

set.

In October, the national press and broadcasters were targeted to reach

parents and children directly. Coverage included The Big Breakfast and

BBC Radio One. Third-party endorsement was secured from the cybernetics

department at the University of Reading, which had examined the

educational value of the set.

The Results

According to CIA, one in six males aged 16 to 30 were reached on average

1.3 times. Metrica reported that 93% of the coverage stated that Lego

Mindstorms was a ’cool’ or ’must have’ product.

The campaign’s impact, which was monitored by NOP, revealed 36% of

parents with 10- to 16-year-olds were aware of the product and one in

three intended to purchase.

Carrick James Childtrack found that 21% of respondents were aware of the

product, 11% had tried it and 73% of them wanted to try it again.

Lego reported that 20,000 units were sold from October to December 1998.

This article was first published on Marketing