Evaluation: Campaign Assessment

In a bid to move away from crude campaign assessments, some industry players are plugging the use of a universal evaluation model. Simon Ellery investigates whether a one-size-fits-all approach would be feasible

How often do you tire of statements such as 'the launch received £15m worth of coverage for a PR spend of £40,000'? Measures such as advertising value equivalents and column inches offer fairly crude but popular methods to gauge campaign success, while journalists can offer views on brand perception.

But leading sections of the industry, backed by the Institute of Public Relations (IPR), want to promote a universal evaluation model.

The interest in evaluation within the industry is huge. The third edition of the IPR toolkit was launched last July and has sold 3,000 copies.

'The IPR has always argued that there should be greater common practice,' says IPR evaluation group chair Chris Genasi. 'However, there is this dichotomy - people are interested in evaluation but find it difficult to understand and do.'

Kaizo director Crispin Manners says the IPR toolkit is great for people who have never done evaluation before, but attacks it for concentrating on media evaluation.

'Not all clients see media as the route to the outcome,' he says, pointing to work the agency did for walking frame manufacturer Zimmer. It wanted to attract orthopaedic surgeons to fill more than 600 places on a training course for a new treatment. The course was oversubscribed. 'I think people should focus all their efforts on approaching the outcome and not just the media coverage to help them get there,' Manners adds.

Porter Novelli assistant director of planning Adam Mack, who was one of those who roadtested the toolkit for PRWeek (5 September, 2003), agrees.

Mack has not used the toolkit since and argues that the industry needs to be more progressive, basing its work on audience or business impact.

He points to work with Hewlett-Packard and its Hype Gallery campaign.

The agency's brief was to drive footfall, and data it collected showed that for the days on and following the biggest PR 'hits', footfall was above the daily average for the Hype Gallery.

The potential problem of a universal model is the variety of PR work in the industry. How can a universal tool have the flexibility to deal with diversified campaigns that range from crisis management, product launches and conferences to media relations, public affairs and corporate social responsibility?

But Genasi defends the drive for a universal system. 'It's a standard that is a bit like a driving test. You have a basic version but that does not stop agencies offering additional versions.' For example, a new evaluation tool was unveiled this month. Freud Communications helped Hall & Partners develop a three-step system that employs an online representative sample to gauge outcome.

'It's a fairly critical time for the industry because it needs to demonstrate its effectiveness or it will remain the poor cousin to advertising and sales promotion,' adds Freud Communications creative director Paul Melody.

Measuring crisis management

One sector in which evaluation remains difficult is crisis management.

When a crisis breaks there is little time to draw up an evaluation programme and, in some circles, good crisis management means minimal column inches.

One seasoned crisis executive adds: 'Having a template can be counter-productive. As soon as there is the suggestion of an issue the press are all over it.'

A universal evaluation system would certainly mean clients could compare like with like, but whether there will ever be a system developed that suits all campaigns still seems a long way off.

EVALUATING AN EVENT

Midnight Communications conducted pre- and post-campaign evaluation for the 2003 Notting Hill Carnival, using a range of tools that included media evaluation software from Media Proof and media archives from Dialog.

Pre-campaign work included a stakeholder audit that provided an overview of some potential strengths, weaknesses, opportunities and threats to the carnival's communications campaign.

Midnight evaluated how negative stories were kept out of the media and the effectiveness of the proactive 'good news' media relations campaign.

Midnight also monitored press coverage on a daily basis throughout the campaign to ensure objectives were being met. Evaluation post-campaign focused on the success of the positive carnival messages and the reduction of negative messages about the event.

The evaluation demonstrated that Midnight had succeeded in keeping negative stories to a minimum by positively promoting the roots of the carnival (its community, food, costumes, music and culture).

EVALUATING A PRODUCT LAUNCH

Ford used Millward Brown Precis (MBP) to assess the impact of the launch of StreetKA. The car giant also wanted to determine the effectiveness of its use of celebrities Kylie Minogue and Julien MacDonald.

MBP's analysis of a broad sample of media found that StreetKA garnered favourable reporting across Ford's target titles. MBP and sister company Mindshare analysed the MacDonald campaign during London Fashion Week.

Evaluation showed that the launch was successful, with StreetKA generating more media attention than nearly any other Ford model - only the Focus got more press.

The association between StreetKA and Kylie was also found to be extremely successful in generating interest in the model, including numerous headlines.

The idea that Kylie represented similar physical attributes to the StreetKA was picked up, and images of her draped across the vehicle were frequently used, reinforcing the connection.

The research reinforced Ford's decision to carry out highly targeted campaigns for its key audience.

EVALUATING CSR

BT took a forensic approach to evaluation for what has been described as its most ambitious social marketing campaign. An IPR award-winning campaign, Am I Listening?, ran in conjunction with charity ChildLine and was supported by socialite Tara Palmer-Tomkinson to amplify the corporate social responsibility work of the BT Social Policy Unit.

Working with Sinclair Mason, BT identified key benchmarks to gauge the campaign's influence on BT CSR reputation and employee perception. Budget was set aside for follow-up research and evaluation of each of these areas.

The first two years of the campaign focused on delivering tangible results for ChildLine through fundraising and strategic and technical support.

Its objectives included ensuring that 75 per cent of BT's 100,000 employees were aware of the campaign, raising £6m for ChildLine and ensuring the views of young people were sought, considered and valued. It also wanted to help the charity answer more calls and boost sales of BT products and services.

Media analysis was provided by Metrica, IMPACON and Cutting Edge, with monitoring services provided by Romeike and TNS. More than £2.5m has been raised, with BT receiving extensive coverage. It counted 1,200 items, generating 212,249,000 weighted opportunities to see, with spontaneous recall of BT's associations with ChildLine jumping 47 per cent by December 2003, according to MORI figures. BT has also increased sales with residential users by four per cent with reduced customer churn, and ChildLine has boosted the number of calls it has answered.

On top of this, 85 per cent of BT's employees are aware of the campaign and 83 per cent believe it has improved the perception of the company.

Have you registered with us yet?

Register now to enjoy more articles and free email bulletins

Register
Already registered?
Sign in

Would you like to post a comment?

Please Sign in or register.