Measurement Roundtable: Successful measures

Metrics strategies have had to evolve as executives demand insight that looks beyond the numbers.

Metrics strategies have had to evolve as executives demand insight that looks beyond the numbers. Bernadette Casey and Danielle Drolet were in New York for this BurrellesLuce-sponsored roundtable discussion.

Bernadette Casey (PRWeek): Are we finding that there is a lot of executive buy-in on measurement? Are conversations happening early enough and is measurement integrated throughout the company?

Allyson Hugley (MSLGroup Americas): Progress is being made toward having it integrated within the company. One of the things I'm seeing a lot more of, even in the past year, is our clients coming to us and wanting counsel and advice on how do we package measurement. How do we package the results so that it is going to be well received by a more senior audience? The thinking is beginning to happen, and maybe over the next year, we're going to see more of that momentum pick up.

Antonio Ortolani (Weber Shandwick): It's very different how a day-to-day PR team would look at measurement versus how a CEO looks at it. CEOs are smart people, but they're busy people. People aren't giving enough thought to how you are presenting data, and getting it to them in the most useful way, as a decision-making tool. CEOs and C-suite people can be a bit frustrated by that. They buy into measurement; I don't think that's an issue. There's still some frustration on their end that this isn't exactly the way they would like to see it.

Natasha Fogel (StrategyOne Edelman): PR has a critical role in the customer buy-in process. PR practitioners are becoming more aware of that. There are those that are still presenting ad value as a way to measure the value of PR, but we are finding more people are open to saying that it's become obsolete, or not as valuable to measure PR. They want to understand how to measure the value of what is being done. They're open to being educated, and then making the change.

Casey (PRWeek): How have the specifics of measurement data evolved and how does that relate to customers' expectations?

Jason Forget (GE Energy): This AVE thing kills us. The purpose of an ad is totally different than the purpose of public relations. An ad is to drive awareness. PR is to persuade people. AVEs work for people because the companies say “Well, we paid for that.” We can actually assign a dollar figure to it, but the purpose to it is totally different. In PR efforts, the end result you're looking for are different things and how you value them is different.

Johna Burke (BurrellesLuce): One of the biggest hurdles probably is that for the storytelling. We're great with those key messages and being able to understand that. On the other side is that the industry is what I call “graphically seduced.” They buy into these beautiful multicolored charts. They're buying into a lot of those quantitative metrics that are available, but they aren't telling the story of what they really mean to their organization. It's even within our own discipline; understanding how to translate and understand the correlation beyond it being a pretty chart. None of the C-suite cares about that. They would rather see something that's generated in Excel that's very basic telling the story about how they're evolving, how they're developing, and what the growth plan is. Early communication about being able to identify those trends is needed, and being able to work on projections based on that.

Hugley (MSLGroup): We just had a number of conversations with clients over the past year trying to transition them away from AVE, but when we think from many of our clients' perspective AVE for better or worse is a critical part of the story that they've been telling. It's a critical part of their trend and their history, and I think there is in some respect resistance to letting that go. We try to introduce other elements of the story, other characters, other players, and bringing tools to the table, but as much as we'd like to remove it from the discussion, it's a reality for many of our clients because it's what their historical story has been.

Donald Wright (Boston University): I wish AVE had an extra letter in there, so we could call it a four-letter word because I really think it's the four-letter word that's haunting us. Part of the problem is certain people use the term AVE to mean one thing, and other people use it to mean something else. If it's used to mean just measures of outputs that makes appropriate sense, but if it's used in terms of the real equivalency or getting into the multiplier effect than it doesn't work.

Forget (GE Energy): The C-suite doesn't necessarily care what metrics you bring in. They expect you to be the expert. You have to be OK with being that expert, saying these are the things we should care about, and this is the number that matters. Come in every single time and say that and they will believe that you are the expert. You're telling them what they should be concerned about, what they should care about. They like the numbers. It helps to justify budget and things like that, but when we have to deal with issues, or when we have to launch a campaign, they really care about the analysis. As an executive, when he or she flies over to this country, and the company has 15 media hits across different channels—print, online, video—they need to know what the conversations are there. The numbers are great, and they can lead you in the right direction. But I'm a big believer in the analysis behind it, too.

Anne Fenice (Yahoo): For some organizations they want to assign a dollar value to the efforts because you know other functions within marketing in particular are doing that. In early discussions, we were getting some pushback from marketing. It was basically debunking AVE, and giving them a really good understanding of the methodology that we are using. They're still not completely adoptive of it, but they're getting it from us, so they're going to use what we give to them. We really need to educate on the fact that PR is somewhat intangible. There's certainly a lot that we can measure, but there's also a fair amount that you can't. How do you show that you killed a huge negative story, and spent a lot of money with your agency doing that and there's nothing on the chart. I think that's another area where PR professionals need to be confident to talk about the fact that it's OK we are not assigning a dollar value next to everything that we.

Fogel (StrategyOne Edelman): We've started categorizing metrics into either a brand health category or into a sales category. If we are tracking sales, what we're trying to look at is are we using primary research to determine the likelihood to buy, or recommend, and look directly at sales data that may correspond to a campaign or brand health, but we also use sentiment and change in perception. I think the CEO has an expectation that is sometimes very different from the CMO, and it's likely that PR folks have to get through the CMO to get to the CEO. Success comes from speaking in a language that the CMO understands. It's putting the knowledge in that linear process, but talking in terms of what the PR goal is in relation to that marketing goal.

Otrolani (Weber Shandwick): It has to roll up to inside recommendations. CEOs are so challenged for time. The point of pretty charts is nice and all, but we all have great data. It's accurate. It's consistent. I think the industry is really moving forward with that. We have some great tools and methods in bringing measurement together. The CEO wants to know what's important here and should this keep me awake at night? They want three great bullet points about what's really affecting their strategy. We also sometimes forget when we're looking at measurement to assess what's the opportunity. We're always so focused on measuring what's there. What isn't there, I find is just as important. That's the kind of insight that CEOs aren't getting enough of.

Wright (Boston University): Half or more of the people in the C-suites have no idea that you can measure PR. It might interesting to try to work with the people who have awards for research of measurement in our field, such as publications and professional organizations like the PRSA and see if they would consider breaking down into research and measurement categories. There could be an award for the best output measure, for the best outcome measure, or an award for the best use of qualitative research and quantitative research. Something like that might push this out further. I think there are some things that we could all do collectively to try to encourage broadening the awards categories.

Casey (PRWeek): What should be the next advancement in practice to elevate the role of measurement in business?

Forget (GE Energy): At the end of the day, it's got to be holistic. It's got to be the whole ball of wax. All the tools I've seen are built around something that's easy to grab onto which is the public discourse, but the next step that I would want to see is helping me build measures that tie together what we've done with employees, that tie together with the press, that tie together with government, and so on.

Fenice (Yahoo): That's the next stage for us, too. It's a matter of pulling it all together. And that's the expectation that the CMO or the CEO will have, how does this all fit together and speak to our overall reputation with all of the different stakeholders as opposed to just these different pieces? If there are obstacles in getting the PR measurement to be really understood at the company, go with someone in marketing who's doing it really well, and just have your piece fit into their piece, and start to tell the story more holistically.

Hugley (MSLGroup): On the agency side, we've always talked about the need to have that first conversation and map out your measurement plan, and define objectives, but increasingly we've had to broaden that discussion beyond what exactly are we responsible for as you would define it in the statement of work, and more broadly, understanding that we may be responsible for media relations, or social media, and someone else might own the website. We need to think about how do we understand the impact we may have had on the traffic there, and who else do we need to connect with within the organization to have the metrics coming in real time, and then we can adjust what we are doing from an external communications standpoint to tie back to what the marketing and digital, and website teams are doing.

Casey (PRWeek): Now that there is a bigger focus on measurement how does that impact the way research is done and the tools and human resources you need to execute effectively?

Wright (Boston University): So many people working in our field are much more communicators than number crunchers. In the major schools, if you go through the research and measurement track, your starting salary will be at least 50% higher.

Everybody realizes that this is something that needs to be done, but I think there are too many organizations trying to do it themselves. They think why should I go and hire someone from this research provider when I can just go out and pay $500 and buy into SurveyMonkey or Zoomerang. Some of the questions these people put in those surveys, I mean the bias is unbelievable. I think we need to be really careful about doing anything that would encourage people to do research on their own.

Hugley (MLSGroup): We go through a lot of rounds of interviewing and vetting people to make sure that they have the right skill set to provide adequate counsel to our account teams, and we discourage them from doing that on their own, and to actually come to a centralized resource. Not that we will do everything, but this is where you need to go through to have your questions vetted, to have someone who can support you, and a team of researchers who can say that's not the appropriate tool.

What we've also seen over the past year is more clients actually coming to us and saying, we want a conversation on your measurement case, but what are the industry's best practices? On the client side there is opportunity, and there are tools and documents that are being developed that are advancing the discussion.

Jeffrey (VMS): Certainly some of the big measurement houses, like us, are totally changing in terms of the staff that we're beginning to hire because the work is much more sophisticated and demanding now. It's hard to keep up with so we're seeing a big change in our whole structure.

Burke (BurrellesLuce): There's that whole other component still where a lot of people feel like they're “doing” measurement. “We use Google Analytics.” Fantastic. You're now taking us back to the Stone Ages. They are all quantitative, and don't have that good sense of what that really means and applying that qualitative component to it, too. There is an art and a science, and it absolutely requires the quantitative and the qualitative.

Ortolani (Weber Shandwick): There is one little potential black cloud in terms of tools, and it's much more in Europe. A big concern there in doing measurement in a day-to-day logistics level is the licensing and publishing rights of the content that you're pulling through, especially in the UK. And in North America, we've been fairly loosey goosey here. No one's really beating down my door, but that day may come. We have all these great tools, developers, and companies sort of riding on that ambiguous space of “I'm going to give you this flood of news from wherever, and not actually pay any royalties.”

Fenice (Yahoo): People are closely watching the New York Times pay wall, too. If that's successful, then we will absolutely start to see a shift.

Forget (GE Energy): On the client side that's massive. That almost became a real sticking point this year. I pay a lot of money for what I get from my vendor, and you tell me “Oh by the way you need to get your own Financial Times subscription and it's going to cost you XYZ.” This isn't going to go backward. It's only going to progress further and further as more people move online and can't get the same ad rates.

Fogel (StrategyOne Edelman): Acquisitions. It seems to be predominantly customer service driven. Zappos started it. It's their customer management model. But more clients, more companies are trying to move toward that customer management model because they're understanding the importance that social media plays. You've got to be monitoring and listening to your stakeholders in these communities and whether you need to engage them in some way, and how you do that there. That seems to be the genesis for what they're doing, and they seem to be doing that really well. The discussion around measurement tools leaves a lot to be desired because of the imperfect nature of the sentiment algorithms and the ability to identify influencers. This is an area where we're going to see a lot of development still.

Casey (PRWeek): Do all the different channels of media play important roles as vehicle for outreach?

Fenice (Yahoo): There's been a whole focus is on social media, but don't forget about the mainstream media. The mainstream media is never going to go away. They're on social media. If you see AP stories, which are the most re-tweeted thing on Twitter, well, who writes the AP stories, right? There needs to be a good balance, and there's been an over-rotation in social media, and now it needs to be rotated back. Even though Twitter broke the Osama Bin Laden news, people were tuning into CNN to understand the coverage and to see perspective, which is important to us to understand how we are resonating and reaching our customers.

Burke (BurrellesLuce): It goes back to the vetted media. The mass media gives some teeth to it. There's some research that's already there, so that's what people trust, as oppose to what they are looking at in social media, which has more of the conjecture around it.

Social media has evolved a little bit faster with that because they are using an API. They're using that data that anybody can get access to and there isn't a copyright to. But things are changing and it looks more like the Financial Times is going to be the first of many that are protecting their content.

People are always going to go to the traditional media for news and to get the overall perspective because Twitter isn't investing hundreds of thousands of dollars for the real research for giving the beginning, middle, and end of the story. For research when you are applying that pressure back to those service providers, it does have to be copyright compliant. You're looking at getting your budget approved, and if you're looking at going with somebody that's going to offer you turn-key copyright compliance for a couple of cents more that's a much better business conversation for you to have then dropping a $250,000 non-negotiable bill on the CFO's desk.

Casey (PRWeek): The Barcelona Principles have done much to help standardize measurement. Has there been overall buy-in for the principles and how will they continue to move the discussion forward?

Wright (Boston University): Those principles weren't just invented that afternoon. They were packaged in such a way that they served as a catalyst as ways to discuss it and the discussion outside of that room has really been stronger than inside the room. Going forward we're going to need case studies in terms of how this was done, and not just the success, but how the public relations person or the measurement person, was able to convince people to do this instead of that.

Fogel (StrategyOne EDELMAN): Within Edelman we created a public engagement roadmap, which goes through some it. We've integrated all of the fundamental steps of measurement, and we used AMAC as a way to support the parts of the process' steps. This is how we are training a new group of people to think about measurement throughout the process, not just at the end. We're putting it into a process that they are already familiar with and is a way to help evoke change so it won't be difficult.

Developing Metrics

Prior to the roundtable, PRWeek and BurrellesLuce hosted a breakfast event featuring Jason Forget, corporate reputation manager at GE Energy. In the Q&A conducted by senior editor Bernadette Casey, Forget says he spends 70% to 80% of his day doing media monitoring and analysis for both the company and its competitors –a new role that was created for him two years ago.

“It's transformed from being very media-focused when I began to using all these metrics and analysis that go along with it to work across multiple parts of the business,” he says.

Now, Forget says his team easily gets a corporate-table seat where they can be more proactive on business strategy.

Hugley (MSLGroup): A critical outcome from AMAC and the Barcelona Principles was it finally took measurement from a conversation and a lot of back and forth to a document, a concrete output. In terms of it being a catalyst, it's not only from individual agencies or practitioners adopting it, but it's also prompted a lot of the other organizations and counsel for public relations firms to think about the role that we can play in that idea of packaging content and putting out stronger points of view about what needs to be happening in terms of guidelines. It was a really positive first step. We are all time pressed to champion this within our organization and the idea of developing tools was a tremendous step.

Terrazas (Ketchum): In terms of educating our agency counterparts, the reception we've had has always been really positive because I think they see the value in all the elements of the Barcelona Principles and not associating the value of PR with AVEs. It's not something that's coming just from the measurement people and measurement geeks in the back room. I think the only area where we get some questions is how they then have those conversations. Really our effort is about educating them so they can have the conversations with their clients.

Forget (GE Energy): I would encourage you not to be too prescriptive of the application of the Barcelona Principles. One of the things I learned very quickly was that, particularly in China, being able to work and adapt to the environment that you are in is critical. I appreciate the need for standardization, but the one thing I learned quickly was that what you think is operating in other places doesn't necessarily work as well somewhere else. It's so critical to be able to implement on the local level.

Casey (PRWeek): What are some of the trends you see coming in the next six to nine months?

Forget (GE Energy): Native language capabilities are both a trend and a challenge. In a business prospective, 60% of our business' revenues are outside of the US, and all of our growth regions are outside of the US and Western Europe. Our infrastructure capabilities in those areas are small at this point, but are growing very quickly, and then the demands on communicators to be able to ramp up as fast as the business is critically important. I can tell you from experience now even in places where we have a long history in non-Latin-based languages, it's really hard.

Ortolani (Weber Shandwick): Everyone's using Google translator. It's fine as a starting point, but it's not going to cut it. We're working on a large global consumer brand, and we're not going to get the level of detail with these automated translations. Especially now where people are telling you, “I know you can do Spanish, but we need to communicate with someone in Ecuador. It's not an every day thing, but when you're dealing with multinationals it's a concern.

Jeffrey (VMS): Because we are doing a lot more global work now, we have found a good solution with doing the measurement in the original language and then uploading English summaries. It keeps you from paying for full translation.

Fenice (Yahoo): The biggest challenge is the “what's next” thing. More and more people are getting their information online through video. No one's really trying to tackle analyzing that yet. That's been happening for a couple of years now and that's going to be huge in the next five years. What's next after that? If you look at the overall population of the globe and who's online, and how people are coming online, those people that are going to be coming online are going to drive significant shifts in the way that people are consuming content and media.

Terrazas (Ketchum): Is there an opportunity with the committees and organization responsible for reaching all those different mediums, to do some media consumption research. Actually talking to consumers and figuring out how a medium fits within the wider space. In the UK, when people worried about TV dying, the Internet association spent a lot of time on how people are consuming media online, what's going on and then they shared that in the industry. I don't know whether the organizations here are doing that kind of thing, but we should be asking them to help us with that.

Fogel (StrategyOne Edelman): Another potential trend would be something around the blend of the insights community, in primary research, social media, and secondary and traditional media—fusing those insights together with predictive analytics. It's using all these great insights that we have, and having modelers that are taking that data, and trying to look back at the past, and anticipate based on what's happened, or what we're hearing right now, asking what do we think is going to happen in the future, and how does that integrate with the analytics function of business intelligence and data warehousing at different corporations.

Hugley (MSLGroup): A trend is the types of talent coming into the organization, to be able to understand the modeling, and the inputs that are really going to matter. You need a more analytical mind than just the qualitative communications side.

Burke (BurrellesLuce): One of the biggest trends and challenges that significantly impacts measurement, is the single screen, such as people who say “Oh no, people aren't buying TVs anymore.” And, that's because people are watching on their laptop or iPad. It's the real estate of that single screen, and then how do you translate that back. So many of us in organizations have a very clear profile of what our consumer looks like, and it's our job to find out how is that consumer interacting with the new interpretation of where they're consuming information - those subtle nuances that absolutely impact the data that we use. It makes the media that matters that much more important to be able to understand what that real estate is and how it impacts those end stakeholders and how does that factor into your equation.

Jeffrey (VMS): Our trend would be globalization. Everyone seems to have a need for the Asian languages, and the Middle East, as well, so we're restructuring and changing to try to meet some of those needs. The second one is we've done a lot of work correlating PR to outcomes, and also to looking at how PR affects advertising effectiveness, and how we deliver that information to our clients.

Have you registered with us yet?

Register now to enjoy more articles and free email bulletins

Already registered?
Sign in