With some justification, not everyone is a fan of survey data. Periodically, a client or prospective client recites one of two quotes. The first quote is from the founder of Apple, Steve Jobs, who said “People don’t know what they want until you show it to them. That’s why I never rely on market research. Our task is to read things that are not yet on the page.” The second quote is from David Ogilvy. “The trouble with market research is that people don’t think what they feel, they don’t say what they think, and they don’t do what they say.”

Those who don’t want to do data collection use the offensive play, which is the Steve Jobs’ quote. The play goes like this; “there is no sense undertaking market research because after all, people don’t know what they want.” On the other hand, support for the Ogilvy position is usually invoked once the unpalatable research findings are tabled. The defensive play switches to “that’s all very well and good, but as we know, respondents don’t do what they say.”

Steve Jobs’ perspective is easiest to reconcile. Jobs was talking about new to the world products such as an MP3 player in 2001. He was right in that, if your product innovation is ostensibly ‘new to the world’ and the market has limited ability to conceive its application, then there is not a lot of value to be derived from asking respondents about adoption. Henry Ford was from the same school when he said, “If I had asked people what they wanted, they would have said faster horses.” However, using survey data is useful for illuminating attitudes, market sizing based on core needs, and understanding current alternatives. If Steve Jobs was talking about an invention that was a phone that incorporated an MP3 player, a camera and GPS then perhaps the market could comprehend such a value-added bundle and more readily estimate adoption. And today, if Apple wanted to understand why users were switching to Samsung, then survey data is surely the best source of data.

The other counter argument is that with respect, you are not Steve Jobs! Every now and then, a visionary entrepreneur comes along, whose imagination and resources are unconstrained, and proposes something as frame breaking as putting “a thousand songs in your pocket.”

For the rest of us – less than visionaries – research is a critical component to optimizing our efforts and investment. However, the rationale for removing insight from the process is different depending on what phase of the research project we are in.

I should tell you, the context in which I usually hear these comments changes depending on where we are in the project. If we are at the end of the project, then the comments often reflect the seminal work by Elisabeth Kübler-Ross. [i] Her five stages of grief model ranges from denial and anger to acceptance and action. There are a handful of alerts researchers receive when the client has slipped into denial and anger. Some of them include questions like “Who commissioned this research? Where did you get the sample from?” And that old chestnut, “You do realise people don’t always do what they say they will do.” We would agree wholeheartedly with the last statement and therefore, apply techniques that sidestep that issue.

As in all fields of life, there is a degree of variability in the quality of market researchers. And as is the case in most (if not all) services, there is asymmetry of information. Meaning the seller is better informed than the buyer, and occasionally the buyer struggles to recognise poor quality researchers from good quality researchers. Here are two basic situations where criticism of findings arising from survey data could be justified.

A third scenario, usually limited to qualitative research for high involvement goods and services (that is, high perceived social risk), is where a social desirability bias creates distortions in stated choice (please see past post – ‘What Colonel Nathan Jessup Could Teach Pollsters’). Similarly, the resulting mismatch between stated and actual can be averted by using inferential analysis to infer importance.

I am not sure about other research companies but, about one in five Forethought colleagues is a marketing scientist. These folks are responsible for design and analysis of survey data. About 60% of our colleagues are consultants arranged into category verticals (banking, telecommunications, healthcare and so on) to maximise domain knowledge.

My job is working with survey data; I love it! Pretty much the whole purpose of my professional life has been to assist clients to predict and effect behavioural change as it relates to marketing. The primary vehicle for that prediction has been, and continues to be, survey data. Forethought operates in just two areas of behavioral change – gaining and retaining market share for services businesses. With over 4,000 projects completed, I cannot recall a single instance where our predictive models have been incorrect. Without a doubt, insights is a game of skill.

Ken Roberts, Chairman Forethought

[1] On Death and Dying. Routledge. Kübler-Ross E (1969).