Monday, October 22, 2012

Poll Watchers Beware

Every presidential election year, I find myself re-addicted to an awesome source of polling data, pollingreport.com. These guys aggregate the raw results of various independent polls and post them in a mostly unexpurgated format. I only wish I could do cross-tabs to break down the results further (e.g., by number of Democrats versus Republicans, age, sex, income, and so forth). Frankly, I find the raw data much more enlightening than much of the terrible commentary. [One notable exception to the usual polling pablum was today's excellent Dianne Rehm show with two experts breaking down the polling into the necessary detail.]

Particularly telling is the number of people who are "unsure" or "refused" as reported in some of these polls. The numbers are as high as 8% in some polls, suggesting that a lot of people are either still undecided, are dedicated to the old-fashioned privacy policy about politics, or are just sick of being asked. Nevertheless, one sees that Obama has quite a lead in a number of these polls when voters are given the option to be unsure.

I often find that business executives want to ignore the "don't know" responses in survey data. I believe they think the results are somehow less meaningful if a lot of respondents don't know the answers. On the contrary, I think executives can learn a lot when people are given the "don't know" option.

For example, when I was on the Paint Consumers Research Program board, we changed the survey to allow respondents to say "don't know" when asked what price they paid for paint. Not only did we get much more accurate results, we discovered that almost half of respondents don't know what they paid, even when the purchase was a month ago or less. From this, I learned that price is a lot less important than I think most paint industry executives think it is. In fact, I believe that price point (low, middle or high in the store's assortment) is probably much more critical in paint buyers' decisions than actual real price. This effect could explain in part why consumers are willing to pay $50 per gallon at Sherwin-Williams when they can get decent paint at $35 per gallon at Lowe's or Home Depot.

Some of the most important decisions in new product development fall to market research interpretation, so I believe everyone involved needs to take a closer look at the results. Surprisingly, for example, the products most likely to succeed are often the products with the most positive responses and the most negative responses. When respondents rate new product ideas, the lack of a strong visceral reaction usually indicates disinterest whereas a strong negative reaction can mean that they have a real interest in the product but are not willing to buy it themselves. A number of market research startups have popped up recently to capitalize on this idea by having respondents design products "for other people" instead of making decisions with themselves in mind.

Perhaps this could be good news for Mitt Romney, whose negative ratings have been going through the roof lately. But not if you subscribe to the idea that real money markets can predict presidential elections. If that is true, our next four years will be Obama's second term.

No comments:

Post a Comment