Measurement Roundtable: Value Assessment

Measurement's role in the PR industry is constantly evolving, especially in the areas of social media and digital.

Measurement's role in the PR industry is constantly evolving, especially in the areas of social media and digital. Erica Iacono and Jaimy Lee were in New York for this BurrellesLuce-sponsored roundtable discussion.

The Participants:
Don Bartholomew, VP of digital research, Fleishman-Hillard
Johna Burke, SVP of marketing, BurrellesLuce
Marianne Eisenmann, head of research, Chandler Chicco
David Rockland, partner, MD, of global director of research, Ketchum
Frank Ovaitt, EVP and head of research, Makovsky and Co.
Israel Mirsky, EVP of emerging media and technology, Porter Novelli
Mark Phillips, VP of communications, USO
Mark Weiner , CEO, Prime Research
Randall Chinchilla, external relations manager for haircare in North America for Procter & Gamble

Erica Iacono (PRWeek): Last year, when we had this conversation, a lot of people were saying that the recession was driving an increase in demand for measurement, having to prove the value of PR, your investment in PR, when fighting for dollars. I'm interested to know if that is still the case and if so, how has that changed your approach to measurement over the past two years?

Marianne Eisenmann (Chandler Chicco): It was the recession in 2001 that really gave rise to our field in a much more specific way because the demand for accountability and the post-Enron era really demanded that level of accountability. Now, X number of years later, is that reinvigorating our industry, which already has quite good momentum? This time around, [there's] maybe not more demand for measurement.

David Rockland (Ketchum): What I've seen is the same consistent growth, year after year after year, with really not much change. It's almost counter-intuitive that measurement is a recession-lagging indicator. You'd think it would be somewhere in the middle. We haven't seen that much of a change except in the last six months. I would say our growth rate has about doubled in the last six months.

Iacono (PRWeek): What do you think that's due to?

Rockland (Ketchum): I don't know whether it's more attention to measurement, whether the conversation around measurement within PR has shifted from, “Oh, we have to do this. It's really important” to “This is just how you do business.” As it becomes a more common part of the lexicon, a more common part of the practice, you're not having the discussion to prove the value of measurement.

Mark Weiner (Prime Research): The genesis is the need to achieve greater efficiency. During the recession, people look at expenditure in a different way, not necessarily to cut back, but to make sure the expenditure that's being made is being made efficiently. And, measurement is a window to that. The kind of work that we're doing may be enabling a company to do what they do better and to take the resources that they have and extend them further.

Don Bartholomew (Fleishman-Hillard): think that social has made measurement more of a base condition for everyone because it's so dynamic. People recognize that you have to get out there and listen, engage, and also measure. But there's still a fundamental disconnect between philosophically as an industry we want to be outcomes-based, there's still not investments in measurement around outcomes. I always ask our counselors: what do you want to happen as a result of this program? It's not just about followers but how engaged you are with the community and how that engagement translates into brand reputation.

Mark Phillips (USO): How is communication measurement within various organizations being rolled into and becoming an integral part of the overall organizational measurement? It's important we do it for our program management, planning, and assessment, but it only really becomes valuable when it becomes integral to the overall business. That's when it's solidified and that's when resources will be put against it and that's when it's expected that you're going to do it.

Weiner (Prime Research): I see it happening with the speed of business and the speed of content. I see it happening in marketing dashboards and business dashboards, where the changes from day to day don't amount to much. But I see it happening more in the case of content, where the content now becomes the daily barometer for how the business is doing as it relates to public relations. I think that having traction with CEOs in a way that represents public relations more than I've seen it at other times.

Rockland (Ketchum): I've seen us as an industry begin to get better about talking the language that the rest of the company measures itself. Our client, Philips – when we put in their measurement system – it eventually at the end converts to a net promoter score. And the promoter score idea is marketing language that the rest of the company happens to use so they can say: this is what the deal is for customer loyalty, for employees, for external communications. What some of the larger companies have started to do is realize that they buy measurement and monitoring from 28 million companies. They [realize]: We're spending way too much money and it's all in different languages. So, I'm going to one provider. They'll tell their agency, “Go get me one provider and get them to talk in the language that the rest of the company talks in.” It's a trend in terms of consolidation of measurement, where companies are making a selection globally.

Bartholomew (Fleishman-Hillard): If you talk the language of the C-suite, you're going to get back to this outcomes place. If you go into a CEO's office and that person asks you “How are we doing in public relations?” and you say, “Great, we got 3.4 million impressions,” there's a total disconnect from the business.

Weiner (Prime Research): Rather than outcomes being the language of the C-suite, I think it's data that's the language of the C-suite. A lot of other C-suite occupants don't necessarily understand PR and impressions and buzz. As long as it's the language of business, and it's data, you show relative performance over time vs. competitors, vs. past performance, vs. objectives, then it's communicating to the C-suite.

Eisenmann (Chandler Chicco): It's about setting KPI. I can cite several examples where several metrics get passed up to the CEO's scorecard and that's based on KPI. One example is message penetration in our pushed coverage. There's a KPI that's supposed to be met every quarter when the scorecard gets developed. The other business units have KPIs as well so it's all about them setting the targets and then how we measure up.

Bartholomew (Fleishman-Hillard): Most CEOs, message penetration is not the language they want to speak. They want net promoter, they want sales-funneled metrics, they want something they understand aligns with the business better than message penetration.

Phillips (USO): Years back, we were dying for data. We're awash in data now so what's meaningful is finding value in the data and then reporting it in a way that resonates with the C-suite, that's in the language of that business.

Weiner (Prime Research): The key is that it's all relative. It's about proving the value of PR to any person that's investing in PR. It's about understanding what their values, what their priorities are, what their preference are for reporting.

Israel Mirsky (Porter Novelli): There's the data language you use to back up whatever … the data is really telling me about how my audience is reacting to my brand and what my message is. They really want [to know] what value has PR brought me this quarter or this year. And, then they want back-up in data form that they can use to feel confident about what you've just said about value. The language you speak is going to be different company by company and that's the disconnect.

Randall Chinchilla (Procter & Gamble): It all leads back to what your objectives are and there's a wide array of ways that you can talk about it. But then that's what forces you to make choices … and then how you elevate that message in a way that is simple, consistent with wherever the business is heading. For us, being consumer centric, we want to make sure that our brands are reaching the consumer therefore measurement is intrinsic to the way we do business. If anything, I will say that the economic downturn hasn't shifted that culture. But, rather that you seeing measurement as a report card of how brands did at the end of the fiscal year, it's more about what do I do with that information, to inform my strategic choices moving forward. The call to action that we're having internally right now is to be far more deliberate than we've been.

Rockland (Ketchum): I would venture that in cases where PR reports into the CMO, you really are talking a language of net promoter score or market mix modeling. If you're reporting up to the chief communications officer, or some sort of function like that, you do get into these more nebulous, more output-driven terms than in a CMO-driven organization, where it's all about market-mix modeling and net promoter score.

Frank Ovaitt (Makovsky and Co.): I had a chief communications officer client, whose CEO has made clear how disappointed he would be if all she is doing is managing corporate communications. It needs to be a part of how we manage this business. Over time, we will see two kinds of CEOs – the ones who feel that way and the ones that will wish they had said that.

Chinchilla (Procter & Gamble): I've seen the change in the industry from six years ago. I've seen all sorts of measurement conversations coming to the table. It's about what is the right thing to track for my business and for my objectives.

Phillips (USO): Is the economic downturn driving that? Or is that the natural maturation of the profession?

Mirsky (Porter Novelli): It's partly technology. We're getting to a point where everything is more measurable and the ongoing move to a larger digital aspect of what we do is driving that because people are used to getting metrics back from an online program. The C-suite understands this and the expectation that previous digital programs have produced for measurability for all effort is dribbling back to us.

Weiner (Prime Research): Technology is a part of it. Maturity of the profession is part of it. I also think the penalties for having a bad reputation have never been clearer. If you look at the headlines in traditional media, with what's happening with BP and Goldman Sachs, it's all about reputation. A CEO would attribute it less to the technology than to that proliferation of information. A lot of research has been done about the impact of a bad reputation – it makes all that work that much harder.

Eisenmann (Chandler Chicco): You still can't discount the overriding business trend of more accountability in a business climate and the changes that have happened.

Iacono (PRWeek): How do you make it a planning tool, instead than just a report card?

Chinchilla (Procter & Gamble): With measurement, our accountability has become higher so there's an expectation that PR will be helping move the needle in sales, year in year out. Every year that goes by, there is a track record that proves for each of the brands we're tracking how we are doing. We are making sure that we are leveraging this information to standardize the way we are talking to our management. It's important for us to be able to benchmark across what we are doing with different brands. Most important is making sure that we are informing our strategy based on what we've learned from the past.

Weiner (Prime Research): Research as part of the planning process is part of the bedrock. Smart PR people use research to develop a strategy. If you want to prove value, an easy way to do that is do a better job and be able to document how you've done a better job.

Chinchilla (Procter & Gamble): Since I've been working in PR, there's been measurement. The fact of the matter is that measurement tools have evolved to become more complex and more detailed and more accessible and therefore, in some ways, more meaningful to management.

Rockland (Ketchum): I think we're all in agreement that measurement is as ubiquitous to public relations as media relations is to public relations. You can argue some media relations is strategic and some is pulling off a stunt to get something in the newspaper that has nothing to do with a client's business. Whether it's media relations or measurement, there are good ways to do it and bad ways to do it.

Iacono (PRWeek): Is it a matter of money that drives how you're using measurement?

Weiner (Prime Research): My experience is that it's a question of willingness, not inability. It's not about budget but a willingness to do it. You can do a lot of measurement very inexpensively.

Eisenmann (Chandler Chicco): Clients will say, “I don't want to spend any of my budget on measurement.” From any perspective, you think: if I don't provide some kind of measurement, how am I going to come back to my client and grow the business and show that we added value and made my client look good in the context of their own organization?

Bartholomew (Fleishman-Hillard): Clients look two ways. They're either looking backward – the how did we do question – or we're increasingly starting to see clients saying, tell me something I don't know. How can we take all of these data sets and how can we create insights before my competitors get those insights? I keep making the point that insight really is 80% human endeavor. We have to use all the data and then some human has to sit down and connect the dots.

Johna Burke (BurrellesLuce): That's one of those qualitative metrics that are more readily available. It makes the qualitative that much more important and makes the human intervention possible. So much of the qualitative data points that you see are like pops and clicks. It doesn't mean anything unless you translate it what actually works for them.

Weiner (Prime Research): There's an emerging risk with the dependence on tools as a surrogate. The bar has been lowered in terms of price of entry and similar monitoring and metrics. People can feel secure in investing in a tool like that but never really get to the heart of what the data is telling them. There's a danger in this kind of easy access to measurement if people don't use it wisely and recognize the unique ability of experts to interpret that data and to provide guidance based on the research.

Phillips (USO): That's what we're wrestling with. How do you get from this mass of data to something that's predictable so that you can have a competitive edge, so you can avoid crises?

Weiner (Prime Research): If you look at what happened with the housing crisis, for example, analysts had the same data. Very few who chose to act on it in a different way and bet that there was a bubble. That's an example where I could think of where some human expertise made a huge difference for some very small percentage of the world.

Mirsky (Porter Novelli): A lot of that was knowing what's in the data, not just taking the numbers at face value but understanding how they were constructed. Until the C-suite really understands and is willing to demand and then investigate that nature of the data collection that's being done, they're not going to be able to trust the strategic recommendations that they'd like to make on the basis of that data.

Chinchilla (Procter & Gamble): The real value comes when you have an alignment on what your business objective is – is it crisis prevention? Is it building reputation? Is it sales – and how you align what the success criteria should be? Likely, the education process is more of a battle unless there is a true organic understanding of how this impacting what you want to achieve.

Iacono (PRWeek): What progress do you think has been made with monitoring and measuring those areas? It seems to be something that is considered a challenge for the industry.

Rockland (Ketchum): Reach relative to the digital space is where the clarity is most lacking. We've got three components of digital measurement – the creators of content, the passers-along of content, and then the readers of content. It's the reach: how many people actually see it, absorb it, and what the effect of that is?

Phillips (USO): If you're working in public policy, doing public affairs, the effect you might want to have might not be immediate. It might be long-term, it might be over the course of changing voting patterns over a number of years, it might be changing the influence you have in the legislative process.

Ovaitt (Makovsky and Co.): It still takes a lot of thought as to how you're going to plan your communications or your relationship-building so it has that link all the way out to action. But this goes back like every other aspect to: what was your objective to start with?

Chinchilla (Procter & Gamble): When we look at digital and social media, we know we need to play in that space so it's not a question mark. We've also recognized that the PR function plays a critical role, particularly when it comes to establishing relationships and conversations that are meaningful. The biggest question mark becomes: how do we track success? One of the biggest gaps we're looking at is even if you are very clear in your objective, there is no standardization or uniform agreement on how to attract reach. Until that's clarified, what we resourced to do is more of an approach of understanding and doing social listening.

Eisenmann (Chandler Chicco): Unless there's an objective or a call to action, it's really difficult. We work mostly in the healthcare space so when you've got a campaign that's nebulous, like disease awareness. How do you measure disease awareness? Sometimes, we're able to build in a call to action, which could be very specific, like go to this link and take a screener. That can be tracked very specifically but that can also become the key measurement that the client is looking at all the time.

Chinchilla (Procter & Gamble): It boils down to what your objective is. If a critical component of your strategy is, in our case it's beauty, to develop relationships with the top 25 bloggers, that's one objective. If that's the case, great. I would still argue that having an outstanding relationship with a specific stakeholder group leads to almost to speaking to ourselves as a function unless I'm able to connect that back to a meaningful business result. When it comes to social, we are at a stage where we are having that gap and it's important that we figure out how to standardize internally how we talk to management and what the value of social media is.

Ovaitt (Makovsky and Co.): Frankly, my relationship with a company that sells me shaving cream or shampoo is very different than with a nonprofit that I choose to support or a life insurance company with which I have a long-term relationship. It's very clear that the consumer companies were out much quicker in terms of how am I going to use social media. It's also very clear that nonprofits got out there very quickly. It's taken an awful lot of the other [b-to-b] companies a lot longer to figure out how are we going to use those, what does it does it do in terms of strategic relationships?

Weiner (Prime Research): It comes back to a constant in public relations, which is about objectives and making sure that the objectives that you set are meaningful, reasonable, and measurable. If you're selling hair care at a million units, or if you're selling power generation turbines at one unit, it's about the relative nature of all of this. It's about negotiating with the people who are investing to get some kind of consensus, some alignment.

Chinchilla (Procter & Gamble): That's part of the reason we're doing this push internally to standardize in some way how we're speaking about success because we have outstanding success stories but they are more in a microcosm or individual brand level, rather than a broad level. Therefore, there is not a consistent way to connect back to overall business objectives. As long as everyone is measuring differently and communicating results in a different way, we're losing a lot of the power behind what the PR function brings to social media.

Rockland (Ketchum): I wonder if, as you look at the future of PR, one of the things you'll negotiate is what is actually called PR. Who owns social media? Who owns internal communications? As all these stakeholders all communicate to each other, you operate in an echo system where everybody talks to everybody in one fashion or another.

Iacono (PRWeek): How do you measure relationships and then how do you translate that into something that a CEO can understand?

Mirsky (Porter Novelli): If you're talking about influencer relationships, that's a much more traditional output. If you're trying to define relationships with the consumer, it's a very different question and breaks down to opinion analysis and trying to bucket huge numbers of opinions from consumers who are communicating about your brand and your products into meaningful groups that you can measure their change over time. At scale, it's a challenge.

Chinchilla (Procter & Gamble): I think about those relationships in a below and above the line type of way. There are relationships that we know from a functional standpoint are critical for the execution of our PR strategy, like relationships with beauty editors, relationships with dermatologists, relationships with celebrities. I don't report as a result that I had a great lunch with a beauty editor. What I have to show for it what manifests to the consumer, which is whether we lock a story or not. That's the tangible or above-the-line result.

Bartholomew (Fleishman-Hillard): Relationships, almost always in PR and always in a business context, are the strategy and not the objective. Fundamentally, measurement is about measuring objectives, and no strategies. Relationships are the means to the end but we probably ought to focus on measuring the end, more so than the means, although it's interesting to do both. We're redefining PR around a much more integrated model. We call it Peso – pained, earned, shared, and owned. That's how we're thinking about metrics. We're not going to wait for someone to say, “PR owns this” [or] “Advertising owns that.” We're saying, “We own all of that.”

Mirsky (Porter Novelli): As soon as you start saying, “I want to own a piece of paid media,” even though it makes perfect sense that you need to feed your awareness funnel of your social program, other agencies start to get very nervous. It implies a different place at the table for the message-building aspect of what you're doing. It implies PR as defining the objective of the message that needs to be communicated with those objectives.

Ovaitt (Makovsky and Co.): In the basic social science underlying public relations, there's a lot of stuff on relationships. What it is? What are the key factors that cause a relationship or not? Those things certainly can be measured by going to the parties and saying, “By what degree did these indicators exist?” There's both an element of understanding the basic research underlying it and when do we use that from a standpoint of measuring the relationship and also from a very practical standpoint of thinking through, “I had this relationship and I need to call on it now. Can I measure the outcome when I call on it?”

Rockland (Ketchum): I wonder if, around eight or nine years ago, the measurement world split and the academic world went toward measuring relationships and the industry world – corporations, agencies, providers – went down the road of managing brand performance and corporate reputation.

Bartholomew (Fleishman-Hillard): If you go to a CEO and ask that same question, “How are we doing in PR?” And you say, “Relationships strengthened and up 6.4% across the board,” you're going to get a blank stare. Having a good relationship may give you the platform in order to create influence. People care about the influence, not the relationships. 

Phillips (USO): It's a false dichotomy. Relationships are important. Relationships are critical. Bottom line, outgrowths, outcomes, sales, donations – whatever it is that directly affects your profit and loss that's also critical. Over the long-term, you can't do one without the other.

Weiner (Prime Research): I used to think that measuring sales and connecting it back to public relations was the Holy Grail. It took me awhile to recognize that once you're accomplished it; I started to recognize that it's really relationships that make for sustainable, profitable engagements. So, it's the relationships. It's the trust and loyalty that relationships build, not in the short-term but in the long-term. Being able to connect PR to sales is really important but then it becomes: what does this really mean? How do we manage it? Relationship-building can be a great facilitator for accelerating sales and extending the life of a relationship.

Rockland (Ketchum): I don't know if there really is a good way to measure relationships.

Mirsky (Porter Novelli): That speaks to the SEO nature of what we're doing, as well. We're not just generating relationships anymore. We're leaving tracks – tracks that people are finding. Every piece of communications that we produce is findable and another consumer is going to be influenced by it, maybe months or years later.

Rockland (Ketchum): Think about how Dell engages with their customer base to help design their own project or products, or how Doritos engages with the consumer to design its Superbowl ad. That engagement somehow measured through social media connections is not relationship management or relationship measurement more than it is brand measurement.

Eisenmann (Chandler Chicco): In different connotation, you want certain stakeholders to take certain action. We measure that very specifically. We know exactly what actions we want them to take. We have a model where we literally show how many have taken this action.

Ovaitt (Makovsky and Co.): I might add to that if I don't feel a particular relationship to the P&G products I use, I darn sure to All of a sudden, there's another factor you have to consider.

Chinchilla (Procter & Gamble): There is no question that in order to be successful at what we do, we need to have outstanding stakeholder relationships. What is the relevance of measurement in that context? So, we do have certain criteria to measure our relationships with our stakeholders. Most PR experts would agree that [measurement] is critical to the work we do. It's intrinsic to what we do as a function.

Weiner (Prime Research): PR is more relationship-oriented. P&G and market mix models. These models continue to show the effectiveness of PR after the campaign ends and that suggests to me that what was happening was more relationship-oriented. With a 2-for-1 offer, there's no relationship-building there at all. The hope is that you buy what you like and you'll stick with it.

Iacono (PRWeek): Where do we stand to make the most progress in measurement? If we're sitting here in five years, what do you hope that we will have accomplished and advanced in the field of measurement? Is it digital and social?

Chinchilla (Procter & Gamble): That's a space that we would like to understand better. One is get a better sense of how to understand the digital world and the impact of what PR plays in it and be able to quantify and automatically link it back to sales. The other one is how do we understand the interdependency and how the lines are blurring between the different types of agencies or functions that are sitting at the table. How do you measure the power of an idea that really travels across all the different touch points, regardless of where it came from? I'm not sure there is going to be a measurement unit for that.

Eisenmann (Chandler Chicco): Maybe there can be an integration measure.

Mirsky (Porter Novelli): That's what Blackberry is doing. They basically put all of their agencies on a level and they're rewarding them on how integrated the campaigns they come up with are. It's a recipe for in-fighting but it's also potentially a way to get your people working together in a much more real way.

Ovaitt (Makovsky and Co.): I would use the term integration in yet another sense, in terms of where I hope we'll be in five years and I think we're making progress now. That is the integration of research and measurement into public relations practice. All too much we get a few people who are assigned to think about that, whether inside the corporation or inside the agency as opposed to we're all asking ourselves as practitioners every day a few key questions that the outcome of which would lead you to use research better, both research coming in and a sense of what other parties are doing.

Rockland (Ketchum): Where was measurement 10 years ago? It was a bunch of people with scissors and Scotch tape and exacto knives making clip books. You could not have predicted this conversation 10 years ago or probably five years ago. We know it's going to be different. I think we'll solve the social media/digital thing but that might not be the thing anymore.

Bartholomew (Fleishman-Hillard): One thing we can be sure of is there's going to be a massive shake-up on the tools side of things. Last time I counted, there were 87 social media listening tools. That number is going to shrink dramatically. You've got people like SAS now, serious enterprise software companies coming in. There's going to be fewer, better tools, I think. On the other side of things, I still don't think we'll have standard metrics for social media, just like we keep arguing about standard metrics for PR.

Mirsky (Porter Novelli): I would guess that the biggest engine in that shake-out is that we're going to see a social media metrics tool from a major search engine. You will see Yahoo or Bing or Google get into the space and create the social media analytics, just like what happened to the Web analytics space after Google came in. I think you'll see hybrid app-store mentality where people are getting the data in one place and they're getting their analysis based on the kinds of objectives they're attempting to measure.

Burke (BurrellesLuce): I think one of the main drivers is where the audiences will go. With all of this diversion of the tools sets and the audience sets, the audiences are going to be more community-based – community wherever that means and whatever their information flow is. That's going to be a driver of what the real information, what the data points are going to be from those audiences, that are going to impact your ability and the desire, depending on if you're b-to-b or b-to-c, to be able to make any predictions or make any evaluations of what that information is. People want to consume the information: are they going to be the people who go behind the pay wall to get it? Or are they going to be the people who are only getting free information?

Weiner (Prime Research): What I'm hearing is that there will be an increased ability to measure in five years. I think still the hurdle is about willingness and desire, rather than ability. That's what's missing now. As long as PR people don't know enough about measurement to satisfy management's questions about proving value and return on investment, we're not going to go anywhere and until those people who invest in PR start to know enough about PR to demand measurement, you're still going to have this dance.

Eisenmann (Chandler Chicco): We have to be careful not to miss the strategy and directional approach. That's really where we, as professionals, need to push our measurement because we're always going to be debating what's the right metric. At the end of the day, we all need to push in one direction, that measurement is advising strategy. It's forward looking. It's directional. It's where we're moving in terms of advising the overall business.

Have you registered with us yet?

Register now to enjoy more articles and free email bulletins

Already registered?
Sign in