It is impossible to read a paper these days without encountering citations of polls, survey results, or focus-group findings.
Increasingly, organizations and their PR advisers are commissioning their own research in an attempt to make a point or generate favorable notice. This is risky business and, as a recent example involving the American Medical Association (AMA) shows, it can easily backfire.
The AMA "Spring Break" survey of 644 women ages 17 to 35 produced headlines across the country, including "Girls Behaving Badly" (Louisville Courier-Journal) and "Girls Go Wild for Booze, Sex" (Boston Herald). Unfortunately, the questionable methodology of the survey led the president of the American Association of Public Opinion Research to say, "[The poll] has no scientific basis... it is silly and should never have seen the light of day."
In his Washington Post media column, Howard Kurtz reported on the flap, noting, "This poll had zero scientific validity." Helped along by the Mystery Pollster blog, the fallout from the story continues to grow.
This was not the first time the AMA used some dubious polling results to promote a laudable cause. In 2001, it commissioned a survey on "college binge drinking" that asked if respondents agreed that "we should stop holding young people solely responsible for heavy drinking and put some of the blame where it belongs - on the alcohol industry." (Two-thirds agreed, by the way.) However, helpful hints, such as "where it belongs," have no place in serious polling.
On that question, and many other "agreement" items, respondents had choices of "strongly agree," "somewhat agree," or "disagree" - giving them two ways to agree (including the middle option, which is usually "neutral") and just one to disagree.
I ran across this "research with an attitude" in a broader review of publicly available survey research on alcohol issues. (Disclosure: My firm has several alcoholic beverage clients.) Here are some other interesting, if not bizarre, findings. Just 6% of Americans admit to "drinking regularly both on weekends and during the week." Is it a coincidence that 6% also admit they've "lied to [their] doctor about the amount of alcohol [they] drink"? Seventy-three percent say they do not regularly drink alcohol with dinner, while 4% drink wine, 8% drink beer, and 4% are "not sure," suggesting they drink too much of something.
Nine out of ten adults say they are concerned about "underage alcohol abuse." Yet 79% say it is unrealistic to expect "that a teenager will not drink alcohol until they are 21." Ten percent of Americans admit to having driven under the influence of alcohol in the past three months. When asked, "How often do you drink alcohol and drive - constantly, frequently, occasionally, seldom, or never?" 91% say "never." Not surprisingly, no one admitted to "constantly" drinking and driving presumably because, sooner or later, they have to stop for gas and/or more alcohol. To make matters worse, those who said they "never" drink and drive were asked a follow-up question: "Why don't you drink and drive?"
Greg Schneiders is a founding partner of Prime Group, a consultancy that specializes in helping clients understand, plan, and execute change. Greg@primegroupllc.com.