RESEARCH & EVALUATION: Total R&E - While a more thorough evaluation process is being embraced by some PROs, there's still some way to go, finds Mary Cowlett

Traditionally, one of the loudest criticisms of PR research and evaluation (R&E) has been that PR folk like to measure success in terms of output rather than outcomes.

Traditionally, one of the loudest criticisms of PR research and evaluation (R&E) has been that PR folk like to measure success in terms of output rather than outcomes.

Research, planning, media monitoring and evaluation have largely been seen as separate functions, rather than as integral parts of an ongoing PR process. PR practitioners have often bolted a few media cuttings and a bit of message evaluation onto the end of their activities to show the success of a campaign.

But, at long last, it seems that this approach is being turned on its head; awareness of the need for the thorough evaluation process to permeate PR activity is up; evaluation models are changing; and R&E specialist suppliers are starting to provide a range of services to paint a more holistic picture of what PR activity has really achieved.

For example last October, following its acquisition by the Swedish-based Observer group, Romeike & Curtice took the opportunity to reflect what it describes as 'a fresh perspective in media monitoring' and rebrand itself as Romeike Media Intelligence.

'By delivering efficient media monitoring and providing qualified analysis, which we call media intelligence, we can increase our clients' ability to make operational and strategic decisions,' says Romeike managing director Bjorn Wide.

'When our extensive experience of media is added to the information we have gathered from print, broadcast and electronic sources, we can become an essential partner to our clients when they are evaluating their publicity and positioning themselves in relation to it,' he adds.

In a practical sense this means that while media monitoring services remain core to the Romeike offering, the company wants to develop partnerships with its clients, to offer more thorough bespoke solutions. In addition, according to Romeike head of analysis Chris Thomas, the company has built up its international capacities, and become multi-skilled.

'Working with a broad framework of industry specific-teams, analysts specialise further in other skills and competences, such as integrating media analysis and market research,' he says.

However, Romeike is not the first to expand its traditional offering to embrace a range of R&E services - and it is unlikely to be the last.

For example, Mantra International has moved away from just traditional media evaluation to a much more detailed service which includes tracking the perceptions of individual journalists, while building a picture of specific issues.

'It's all about making communications more effective and accountable,' says chairman Brian Moore. 'A public's contact with a client comes not only through PR, but also through advertising, promotions and referrals.

And then there are the customer's actual experiences with the client.

All of this combines to help build a perception of a brand or company.'

Most recently, Moore's organisation has been working with Cap Gemini Ernst & Young on a European Union project in the Benelux region, ensuring synergy between outgoing messages and literature on the single currency for small businesses. 'The critical issue is that each part of research can be carried out and used in isolation, and at the same time be integrated to produce an holistic communications audit,' says Moore.

Counting the cost

One of the worries for PROs considering a sophisticated R&E programme is that it will cost an arm and a leg - especially since most clients still aren't spending anywhere near the five per cent spend on R&E recommended in the Research and Evaluation Toolkit.

But according to Mark Westaby, Metrica managing director and the Association of Media Evaluation Companies (AMEC) chairman, it doesn't have to be that expensive. He reckons there are plenty of off-the-shelf products retailing at pounds 2,000 or under for those with small PR budgets. He also thinks that clients who blanch at dedicating a chunk of a pounds 50,000 PR budget - the current industry average - are looking at matters the wrong way round.

'For an organisation like a bank or a supermarket, pounds 10,000 is peanuts' he says. 'The point is not to set that figure against the PR budget, but against the price of your corporate reputation. People should be asking themselves: 'If I'm spending pounds 50,000 on a programme, what is the price of getting that wrong?' The emphasis should be on research as a planning tool.'

Perhaps understandably, as chairman of AMEC, Westaby feels research providers have grasped the idea that R&E should be part of the same continuous process as planning better than the PR industry. 'I think the market is ready to give PR what it needs - my concern is whether the PR industry is ready for it,' he says.

But there are signs that an integrated approach to R&E is finding increasing favour with many organisations. 'We work in quite a niche area, providing long-term care insurance,' says Paul Bennett, marketing manager of Bristol-based PPP Lifetime Care. 'Our marketing needs are tailored to PR because we are trying to build mass market appeal through education.'

His organisation is currently building awareness with two distinct audiences, financial advisers and consumers. With key messages revolving around working in partnership with the government on certain issues and establishing itself as the sector leader, PPP undertakes media monitoring to see how it stacks up against its own objectives and its competitors.

'It's a way of joining the circle and filling the gaps, so that we can use those results to help improve our planning,' says Bennett. 'For example, it's not just a case of tracking which journalists are writing on a subject, it's equally as important to work out who's not writing on a subject, so that we can improve our contact programme,' he says. 'It also ensures that we are delivering messages that are constantly evolving,' he adds.

Using such techniques also means that companies can take a more sophisticated, fine-tuned approach to their overall marketing, using different elements to a greater and lesser extent according to need.

Last year, MORI sister agency Test Research carried out a tracking programme on Barclays Bank, studying its PR alongside its 'Big' advertising campaign.

'We were able to show that as people's exposure to Barclays advertising increased, so its image also changed,' says Test Research managing director Tim Burns.

In fact, this demonstrated that a disastrous PR policy - closing local branches and hotly contesting ATM charges - was being aggravated by an insensitive advertising campaign.

Barclays did not commission this research, but Burns says: 'It demonstrates how you can look at media exposure and find out how it is affecting people and how communications work, or you can look at PR in isolation.'

But if the quality of R&E techniques available to the PR industry is improving, who is applying the pressure to use them? With the exception of a few shining examples on the agency side, most feel that the wind of change is being driven by the top marketers and board directors in client companies.

Under pressure

Last summer, the all-industry PRE-fix initiative (see p17), commissioned MORI research of this target group to ascertain attitudes and perceptions towards R&E and explore their needs. This revealed that PR decision-makers feel pressure from the board to show results; from the purchasing and cost controllers; from the planning process and from themselves - 'we feel on the back foot'.

However, there were some perceived difficulties and limitations - not least that, as PR covers such a broad spectrum of activities, one size will never fit all. Many also felt that evaluation was not always possible.

'How do you evaluate bad stories kept out of the media?' asked one respondent.

At Echo Research, which offers services from media content analysis to reputation analysis and crisis management, CEO Sandra Macleod thinks companies should be more prepared to use research to examine the bigger picture and to inform strategic decisions.

'With listed companies especially now having to look closely at social and environmental issues, we can do a lot in terms of being the litmus paper for changing sentiments,' she says.

'What is the research telling you about the drivers of your reputation?

What are you doing well that you should do more of? What do you need to change?' she asks.

The signs are that by working in partnership, all the stakeholders in the debate are beginning to make huge strides in a more well-rounded use of research and evaluation.

But there is still a long way to go. Countrywide Porter Novelli planning director Paul Miller says: 'I think there has been progress in horizontal integration of research and analysis of information, but it is still largely in the domain of media content.'

PROs have embraced more sophisticated media evaluation, looking at trends over time and drawing performance comparisons with competitors. The areas that now need work include bringing business performance measures, audience awareness and customer behaviour patterns into the frame.


PRE-fix, the all-industry initiative promoting the benefits of planning, research and evaluation (PRE) as an integral part of PR programmes, was launched last October.

It is aimed at building on existing industry initiatives such as PRWeek's Proof campaign and the Research and Evaluation Toolkit.

Chaired by Propeller Group managing director Martin Loat, with Chris Genasi, CEO of Shandwick Corporate representing the IPR and Carrot Communications managing director Richard Houghton representing the PRCA, the PRE-fix team intend to provide information and resources to PR decision-makers in a practical format.

Initially, this will be in the shape of a website, which PRE-fix PR consultant Nick Fitzherbert is putting together for launch this spring. With a complete listing of services from various R&E providers and a handful of case studies, his aim is to provide PR decision-makers with guidance on which methods suit which situations at which points in time, with some illuminating case studies.

PRE-fix intends to keep the advice short and sweet. 'Compared with the Toolkit, this will be more in the style of a Swiss Army Knife; it will be easy to use, versatile and compact,' says Loat.

Fitzherbert has some ambitious plans for ensuring that the PRE-fix initiative shakes up the R&E debate. 'It's up to me to find lots of opportunities to tell the world what we are doing, so there will be a media programme,' he says. And rather than relying solely on historical case studies, he intends to grab journalists' attention by applying evaluation techniques to current news stories, especially business crises.


Internet site Fish4 was launched in October 1999 with backing from the UK's regional press, including Trinity Mirror and the Guardian Media Group.

As the entry point to a range of online services, it provides access to Fish4cars, the largest database of used cars in the country, Fish4jobs which holds up to 30,000 jobs and Fish4homes listing around 30,000 properties.

Through its online directory, Fish4 also features 1.9 million local businesses.

In the true spirit of the internet, the aim of Fish4 is to bring together a wealth of national data, while positioning itself as the 'local expert'.

'In theory, the logistics involved in delivering such highly-localised information are nightmarish,' says Fish4 sales and marketing director Jonathan Lines. 'But in practical terms, we have access to unsurpassed and extensive content drawn from around 80 per cent of the UK's local print media.'

As an internet brand, with few assets save its reputation, Fish4 undertakes a huge amount of research to ensure success and evolution as a company.

'We have to know how we are perceived across all our target audiences,' says Lines.

For the most part, this work is conducted in partnership with Echo Research, which evaluates brand recall and awareness, the effectiveness of above-the-line advertising spend, the impact of PR activity and Fish4's share of voice in the media.

By correlating these findings with data gathered from consumer interaction on the websites, Echo is then able to demonstrate the impact of Fish4's overall marketing and PR efforts and help ensure that budgets are working to best effect.

Line says: 'We rely on research a great deal and are continuously commissioning new projects all the time. Working with Echo allows us to undertake several different types of research with one company which already understands our brand values and objectives. In the time-critical world of an internet business, this kind of support is highly valuable.'

For Lines, a major achievement of this research and ongoing evaluation process has been the swift, fine tuning of PR activity his company has undertaken in response. 'We were able to see gaps and take steps to alter course pretty much instantly,' he explains. 'In a competitive environment such as ours, this is a major asset.'

Have you registered with us yet?

Register now to enjoy more articles and free email bulletins

Already registered?
Sign in