One Part Data, One Part Storytelling
Four out of five dentists surveyed recommend sugarless gums for their patients who chew gum.
I grew up hearing this marketing quote and I admit, it was compelling. The nice folks at Trident wanted me to believe that those four dentists were actually recommending Trident gum, not just any old sugarless gum. But the important take-away for me was that there is a story to be told in data.
A spreadsheet full of data can seem mind-numbing until I start to wonder what it means. One of the stories that Trident wanted to tell was the quote at the top of this article. It's a good story...it sticks with me all these years later (due in part to how long and often they used it in marketing). But in the years since, I've looked a little more closely at a lot of survey data. It turns out there are TONS of stories to tell from survey data, some of which even seem contradictory. So my advice is, as for many things, buyer beware. Before you buy the story someone is selling from their survey data, you should ask a few questions:
When was the survey done?
Old data can be worse than no data. Is it this year? Last year? 10 years ago? A 10 year-old survey about personal technology preferences can likely be ignored since iPhones (for example) didn't even exist 10 years ago and the impact of smartphones on personal technology seems like maybe a big deal. Conversely, a 50 year-old survey about sugarless gum may not have changed much. Although to be sure, we'd have to ask the four dentists, which makes me wonder...
Who did they ask?
Who was the 5th dentist? More importantly, who were the other four? Knowing the survey audience helps you understand why they might reply a certain way. The gum survey could have included dentists just out of college, tenured dentists, or both; dentists from all over the country or in a concentrated geographic area; or worse they could have been (by chance or diabolical design), a population of gum-chewing dentists. Personally I suspect there are more gum-chewing dentists who would recommend gum for their patients than non-gum-chewing dentists, which makes me wonder...
What did they ask?
The specific wording of the questions matters. A poorly written question may have more than one variable, may be ambiguous, or may lead the respondent. So while it's a little wacky to wonder what they asked so you can tell your own story, it's helpful to know the questions before you completely buy their story. The context of these two questions are important:
What didn't they ask?
Some questions are deliberately avoided. When I worked for a wireless company, we often surveyed customers but we never asked them whether they were happy with the coverage in their area. We avoided this question for two reasons:
What story would you tell?
So far I've given you nothing but cautionary tales and none of the pure unadulterated joy that is survey data with solid demographics! I have personally spent many happy hours wondering what people in the northeast US thought compared to people in the western states; wondering what issues older folks and younger folks agree on; and what a new person (to a product, or a company) thinks compared to a tenured person. And in consideration of the fun that can be had with a data set, I give you this gift:
On an average day in 2013, employed adults living in households with no children under age 18 engaged in leisure activities for 4.5 hours, about an hour more than employed adults living with a child under age 6, the U.S. Bureau of Labor Statistics reported.
The wonderful folks at the Bureau of Labor statistics make this and tons of other survey data available to you, the taxpayer, just because they have it. And even as you read that quote, I hope you start to come up with stories about households with children and those without. So take some of that leisure time (childed or non-childed) and tell some stories. You're welcome.
(This blog post is re-purposed from an article I wrote on LinkedIn in 2015)