Take the best of both worlds

The boundaries between market research and evaluation are being broken down by data-driven clients, says Alexandra Jardine

It is a familiar scenario: a PR firm puts in place a spectacular consumer campaign for a new product or service, resulting in masses of positive editorial coverage. A few months later sales soar.

The PRO may well be very happy - he has proof that his efforts have succeeded with journalists - but the client wants more: analysis showing how the target audience has been influenced.

Two types of company attempt to offer PROs this evidence: media evaluation firms, which started life as monitoring organisations; and market research agencies. Traditionally there has been a well-defined distinction between the two. Media evaluation firms measured a PR campaign's physical results - the amount and type of coverage it gained in the media. Market researchers, meanwhile, measured the impact of the campaign on public opinion.

Now, however, clients are increasingly interested in the 'provable' end results of PR and want to link these disciplines - to bridge the gap in the evaluation process.

Complementary disciplines
For media evaluation companies the days of simply totting up column inches are over and measurement methods have become more sophisticated, often edging into market research territory.  At the same time, some market research specialists are seeing the benefit of offering media evaluation services. So, is the distinction between the two blurring?

Philip Lynch, a director at monitoring firm Presswatch Media, says: 'Clients need to understand how they are perceived and represented by the media, so detailed evaluation of print and broadcast coverage will continue. That said, for a consumer campaign to be successful, it must register with the target audience, so analysing public opinion seems a logical step in the evaluation process. These boundaries are definitely shifting, especially in consumer PR.'

James Davies, managing director of Impact Evaluation, says both aspects of evaluation are vital: 'You need to know where you are - that's the audience research part. But you also need to know how you got there - by evaluating  media coverage. That's the only way you can improve.'

But while the marriage of output (coverage) with outcome (public opinion/sales) might sound like common sense, it has not always been that simple. Historically, PR departments have commissioned media evaluation, with marketing teams overseeing market research. For many brands this means that post-campaign analysis of coverage is rarely, or never, linked to sales data or market research - unlike advertising, which is always given hard figures to prove its worth.

Making a connection
The situation, however, is changing. Peter Christopherson,  consumer and technology director at Echo Research, says: 'There are some organisations that can clearly see the connection between the disciplines: that it's not just what appears that counts but the impact of that coverage as well.'

These clients tend to be larger corporations where PR and marketing departments are well integrated. And according to Davies, it is therefore in-house PROs, rather than PR agencies, who are at the forefront of comprehensive evaluation, mainly because they have the resources to do it.

This is an important consideration. While the PR budgets given to hired agencies often stretch to media evaluation, market research is usually deemed too expensive for inclusion: not many companies are going to spend as much on evaluation as was spent on the campaign itself.

So how should agencies spend their budgets? Guy Washer, managing director of Benchmark Research, says that although many agencies are reluctant to pay for market research, it has much more kudos than media evaluation and will benefit them in terms of proof of worth. 'People don't tend to look at monitoring data: it's just a tick in the box,' he argues.

Benchmark, for example, evaluates the effects of PR campaigns by polling 'influencers' such as journalists and analysts. The respondents might be questioned on their awareness, spontaneous or prompted, of brands and the associations these products or services inspire.

Edward Bird, evaluation director at Romeike and chairman of the Association of Media Evaluation Companies, says evaluation firms are offering PR agencies and their clients more cost-effective ways of demonstrating value, by 'thinking more creatively about the relationship between output and outcome', thus encroaching on market researchers' territory.
'The challenge to market research is that the evaluation side is becoming more sophisticated,' he adds, pointing out that basic market research sometimes fails to reveal which media sources influenced public opinion.

Sophisticated analysis
Romeike's analysts, who used to simply measure column inches and tone, now try to give context to their research. 'We are moving more into a consultative role, becoming more involved at the planning stage [of campaigns] and looking at where to target them,' says Bird. 'Some of our evaluation measures are being used by clients as performance indicators - a shortcut that tells you how the business is doing.'

The British Airport Authority's Airport Express division worked with Romeike last year,on the campaign for its Gatwick and Stansted Express services and the newly launched Heathrow Connect. Romeike tracked the volume and tone of media coverage by publication, journalist, spokesperson and external commentator, using a 'five-point favourability scale'. It found that while the positive outweighed the negative, campaign messages were not always successfully delivered. Bird says that Airport Express now uses this evaluation to inform its comms strategy.

Another evaluation company, Echo Research, has worked with clients including the BBC, Standard Life Investments and Sainsbury's. When the supermarket required an assessment of media coverage for the launch of its autumn/winter brochure in 2004, Echo tracked and rated more than 600 print and broadcast items. It analysed the use of images, assessed which press releases led to coverage, and examined whether journalists' attendance at a  media event could be directly linked with coverage. It also calculated audience reach.

Echo's Christopherson says his firm's findings allowed Sainsbury's to refine its PR strategy for the following year by identifying areas of strength and weakness.

However, he adds that while Echo works with established market research companies to assess the impact of campaigns on consumers, he does not see a day when Echo would conduct both research disciplines in-house. Nevertheless, some research companies are going down the in-house route. For example, Impact Evaluation has seen increasing demand from clients to - this is its own phrase - 'go into the audience', and is beginning to conduct its own market research via freelancers.

Elsewhere, Presswatch buys in specialist market researchers to supplement its media evaluation work. This arrangement, says Lynch, allows it to tailor the market research element to meet clients' specific requirements. 'I can see more formal tie-ups between evaluation agencies and market research firms if the demand is there and clients are willing to pay for additional services,' he explains.

Dan Naylor, director of sales and client services at Durrants, says the company's evaluation techniques have evolved over the past few years to include qualitative analysis, with analysts measuring aspects of coverage such as favourability, message content and tone. He acknowledges that the addition of market research would 'close the loop' between media evaluation and its sister discipline.

'We recognise that the requirement of measuring press cuttings can only take you so far,' says Naylor. 'People are looking more at what actually happens as a result of coverage. We are already looking at how the different elements of the PR footprint interact with each other - integrating planning, monitoring, evaluation and reporting. Market research is on the horizon.'

If media evaluation companies are making inroads into the market research sector, it is also true that market research firms have recognised the role that media evaluation plays.

Two-way action
In 2002, market research company Taylor Nelson Sofres set up a media evaluation arm, TNS Media Intelligence, after acquiring several specialist evaluation companies around the world. Dean Wading, deputy managing director at TNS, says its philosophy is the integration of market research and evaluation.

'Companies need to get tangible feedback as to whether their PR has worked. Just telling them whether or not they have reached certain numbers of people is not always the best way to go about it,' he says. 'The only real way to find out whether a campaign is effective is  to look at sales or share price.'

Wading adds that some PR agency clients are adopting a 'from cradle to grave' approach to analysis. In these cases, TNS media analysts and market researchers meet to discuss linking coverage evaluation with sales results/public opinion surveys. A sample of the target audience is polled on its exposure to the PR campaign in question, and this is measured against whatever other activity is running, such as advertising, to determine whether a correlation exists between behaviour and the PR campaign.

Clients using this service have included a large pharmaceutical company that conducted research, both before and after a media campaign, for a drug's shift to over-the-counter availability.

While it is the PR teams of companies that are largely driving this demand for sophisticated evaluation, PR agencies certainly appreciate the benefits of the research market. At Ketchum, global director of research David Rockland heads the agency's evaluation unit in New York. He says the agency is better placed than many evaluation companies to advise PROs on how to use research. 'It used to be a case of handing over a 100-page report, but now clients need that boiled down into an explanation of the consequences for their business. We can say: here is the data, now here is what you should do with it.'

The agency also works with third-party market research groups on certain projects. It is currently working for an organic food company alongside Milward Brown, for example, on a consumer market research questionnaire on attitudes to organic produce.

'I see those connections as the way forward,' explains Rockland. 'In many situations it's a case of adding a PR dimension to the tracking work already being done by a company.'

Useful equations
Rockland says he is less worried about sharing work with specialist market researchers than he is about clients not seeing the value of PR measurement. 'My biggest concern is the loss of business from PR to other communications disciplines. So even if the client is also working with an evaluation firm, at least we can reveal how PR fits into the equation.'

He makes an important point: while PROs might naturally be wary of adopting market research - traditionally the preserve of the marketing department - any activity that shows a clearer line between PR and results is surely helpful.

'PR teams are increasingly being asked to perform in a business-oriented way, and PROs are being put under the same performance management targets as other parts of the business,' says Romeike's Bird. 'If you can show the relationship, for example, between PR and sales, it will be beneficial.'

Presswatch's Lynch concludes: 'Greater use of market research will promote a more sophisticated understanding of audiences, and this can only be a good thing for PROs, who can use this knowledge to devise more effective campaigns.

'It may also make marketing managers more appreciative of public relations as a channel for connecting with customers - recognition which is long overdue.'


Over the past few years the BBC's portrayal in the media has become more important to the corporation. As well as changes in the way that the public use media to find out about programming, it has also had to deal with challenges to its corporate reputation.

Recent issues include the Butler Report on the BBC's handling of the David Kelly affair, the appointment of Mark Thompson as director-general, consultation on its Charter renewal, job cuts and a journalist strike. 

In January 2004 the broadcaster brought in Echo Research to establish a media evaluation programme, which is used by the corporation as guidance on its overall communications strategy. It also acts as a tactical tool for those dealing with journalists.

Echo produces a monthly report, which is circulated to the BBC's senior management and its press and media teams. The main areas it details are corporate coverage and programme publicity.

The report is designed to give an overview of the impact of BBC press coverage by linking news and feature coverage, as well as programme publicity, back to the corporation's strategy. For example, it includes feedback for publicists on how successful they have been in linking programme messages to core organisational ones.

It also gives an impression of the success of campaigns in terms of their target audience reach. Examples include assessing programme publicity for the new Doctor Who, the second series of which stars David Tennant alongside Billie Piper. Publicity for the first series, which aired last year, was deemed to have been a major success.

Greater insight
Echo also monitors debates on subjects that affect the BBC, such as Charter renewal, and tracks journalists for their stance on the company.

According to Mark Kelleher, senior communications manager at the BBC, the report is used to shape the corporation's overall strategy.

'We can measure press coverage against our objectives, and it gives much greater insight into how we manage the different messages coming out of the BBC every day,' he says.

Jonathan Hughes, joint managing director, Golin Harris

Q) Will it help if you can buy your market research and media evaluation from one place?

A) Sure, as long as they are treated as distinct specialisms and the company has qualified experts in each. It will be interesting to see how this plays out, and I would expect the market research firms to be better placed to develop media evaluation services than the other way round.

Q) Do you feel that suppliers, by broadening their expertise, will dilute their skills, or will reporting standards in PR finally improve?

A) It's not a problem if the disciplines are kept distinct. However, I'm not sure that the business case exists for this. There is a huge investment required to set up and maintain a media monitoring and evaluation service, and the profit margins are far greater in market research. Perhaps it would make more sense to have affiliate relationships, so there can be a single point of billing, quality control and preferential rates, while keeping the expertise separate.  

Improving reporting standards is not reliant on suppliers broadening their services. That's like saying better driving is dependent on better cars. The services already exist - it's up to the PR industry to get its act together.

Q Is media evaluation more important than market research?

A) No. If you're at the beginning or end of a campaign, use market research to identify how attitudes have changed. Use media analysis during a campaign to enable you to fine tune it.

Marilyn Baxter,  non-executive director of the Central Office of Information, on behalf of the Market Research Society (MRS)

'The accepted way of evaluating PR is only a small part of the whole picture,' says Baxter. 'The pressure for more sophisticated evaluation is coming from market research-based companies, which have been used for a long time in areas such as advertising, but not in PR.'

Baxter points out that historically, 'evaluation' of PR has meant simply looking at the amount of coverage generated by a campaign for a particular product or service. 'But evaluation of other activity, such as advertising, would look at whether the activity actually worked,' she says.

She explains that the traditional PR evaluation companies have been forced to play catch-up with the market researchers: 'The whole approach to researching PR is changing. People need to look at results and ask not only whether a campaign worked, but whether it worked in the way they expected and how. Then they need to learn from what they did and build on that.'

Baxter says the push is coming not so much from PROs as from marketing directors, who are much more interested in evaluation of company-wide activities. 'PR is becoming much more used as part of an integrated communications plan, and is becoming more strategic,' she says. Baxter argues that if PR activity is integrated into the marketing remit, the barrier of PR budgets not stretching to market research could be overcome.

She adds that the growth of online polling is lowering the cost of research.

The Market Research Society and the British Market Research Association (BMRA) are planning to join forces by integrating their services.

According to the organisations, the move will achieve unity of purpose in the sector. The proposal is being put to a vote of BMRA members early this year.

Have you registered with us yet?

Register now to enjoy more articles and free email bulletins

Already registered?
Sign in