When we first started doing research at Enquiro into how people used search, we found very quickly that what people say and what people do are very different things. It just happened that we were doing a survey and a focus group at roughly the same time. In the survey, where we got the results first, we asked if things like the position of a listing was important in whether people read it or not.
 |
| What’s Wrong with Market Research |
We asked people to rank a number of factors on their relative importance, including position, relevancy and trust in brands and vendors shown. Almost without exception, in the survey, people indicated that relevancy was the key factor. They also indicated that they read listings pretty carefully and gave a fair amount of thought before selecting one. Finally, many said they would never click on a paid listing.
Then, we invited about 30 people into our labs and actually recorded their interactions with the search engines (before our eye tracking studies) and it quickly became obvious that how they said they used a search engine and how they actually did were two different things. The vast majority of clicks happened in the first few listings. Many who indicated they wouldnt click on paid listings actually did, and perhaps, most interestingly, the average interaction was around 10 seconds or so. Subsequently, weve seen this type of behavior repeated in eye tracking after eye tracking study. Of course, the famous golden triangle study we did with Eyetools and Did It, and subsequent ones conducted by Enquiro, have shown over and over how quickly we interact with a search engine and how much of our scanning activity is top loaded. Also, we dont really skip over sponsored listings, but in some circumstances (research based activity) were less likely to click on them. Weve used this body of research to come up with a fairly consistent model of how people interact with search results. The results belie what people indicated in our very first survey. Well over 60% of the clicks happened in the first 4 or 5 listings, including the top sponsored ones. People generally spent just a few seconds on the page (around 10 to 12 seems to be the average) in which they scan (not read) 4 to 5 listings. There was almost no deliberation. People click quickly, and if they dont like what they see, they click back. It would take the average person about 2 minutes to actually read all the results on the average search results page. Even if we just read the top 4 or 5, we’d be spending about 30 to 40 seconds on the page. It takes about 7 seconds to read one listing. But we don’t spend much longer than this covering 4 to 5 listings, about 2 seconds per listing. Obviously, we dont give a lot of thought to the credibility of the search listings.
So, were all 1600 of our original survey respondents liars? Were they intentionally misleading us? No, they were just being human.
What we found was the systemic fault with almost all market research. And theres a very good explanation for it. Were generally not aware of 95% of what we do or why we do it. Thats because much or what we do is hidden in our subconscious. Im currently reading How Customers Think by Gerald Zaltman and he pinpoints the problem with traditional market research. In almost every case, we ask people to tell us, either verbally or through writing, what theyre thinking. Just by doing this, we kick in the cortex, the rational seat of our intellect. But Zaltman tells us that at least 95% of every decision is made subconsciously. There, in the murky depths of our brains, predating the evolution of our cortex by many millions of years, thoughts are created through tremendously complex connections of memories, beliefs, instincts and intuition. In many cases, our decisions are made long before they bubble up to our conscious minds. The conscious mind exists to put a little polish on them and, in most cases, to rationalize a decision that was largely based on primal instincts. We may have done what we did because our flight or fight mechanism kicked in, or because our need to procreate surfaced. Thats why we chose the minivan, or the red convertible. It really had nothing to do with the Consumer Reports rating. But, being highly evolved humans, we convince ourselves that our choices are much more rational than those of a lizard (our basic brain core, which rules many of our decisions, is basically the same as a reptiles brain).
In our case, our initial respondents indicated that they deliberated over which search result they chose. In actual fact, there was little risk in choosing a wrong link (its not like our lives, our family or our money is at stake), so we cut off the amount of deliberation we did and after a quick scan, picked the result that seemed to be most relevant to our intent. The lack of deliberation wasnt lack of intelligence, it was a survival instinct bred into us by eons of evolutionary refinement. If theres no immediate risk to us, why should we kick in our brains and spend unnecessary time and cortex processing power to come to the optimal decision. Its not required. A simple scan and click will suffice. Our brains are simply doing what theyve been programmed to do. And its not that the decisions are bad. As Malcolm Gladwell shows in Blink, often these decisions prove to be better than the ones that we endlessly deliberate over. Our brains, especially the 95% that remains under the surface, are amazingly adept at making good decisions.
But theres a more fundamental issue here. If what we experienced in search is typical in all market research (which it is) how do we ever find out how people actually make purchase decisions?
This is a significant challenge, the extent of which might not be obvious at first glance. Let me use an analogy to further illustrate. Remember the tale of the shoemaker and the elves? Let me use that and adapt it slightly for my purposes. For those of you unfamiliar with the story, a poor shoemaker only has enough leather left for one pair of shoes. He cuts the leather and lays it out for stitching the next morning. He awakes, amazed to find the shoes made, and meticulously crafted at that. Elves apparently helped out during the night, soon to bring fame and fortune to the shoemaker.
But what if the elves didnt exist. What if, instead, the shoemaker was actually making the shoes in his sleep? The idea is not so ridiculous. Rumor has it that Coleridge actually wrote Kubla Khan during a dream, and managed to scribble it down before it faded from his consciousness. As any psychiatrist will tell you, were closest to our subsconscious when were hovering between sleep and wakefulness. Its about the only time we get a glimpse into those murky depths.
So lets say our shoemaker actually makes the shoes in some bizarre bout of sleepwalking. He awakes every morning, to find the shoes nearly perfectly finished. All he needs to do is add the laces and a bit of polish. And the shoes are fair more carefully crafted then he could ever accomplish while awake.
The shoemaker really isnt aware of where the shoes come from. In fact, as time goes on, and as he receives more and more recognition for the quality of his workmanship, he begins to believe that its solely due to the little bit of work he does while hes awake, threading the laces and adding a little polish. He learns to ignore the 95% of the work thats done while hes asleep.
Now, imagine someone comes to ask him why his shoes are so exceptionally crafted. Would he admit the truth and say he doesnt know? No, pride and genuine lack of knowledge would keep him from saying that. He has no idea what he does while hes asleep. Its almost as if someone else did the work for him. His conscious brain would kick in and come up with some perfectly rational but completely untrue explanation. Clotaire Rapaille, in his book The Culture Code, cites an example of this:
In a classic study, the nineteenth-century scientist Jean-Martin Charcot hypnotized a female patient, handed her an umbrella, and asked her to open it. After this, he slowly brought the woman out of her hypnotic state. When she came to, she was surprised by the object she held in her hand. Charcot then asked her why she was carrying an open umbrella indoors. The woman was utterly confused by the question. She of course had no idea of what she had been through and no memories of Charcot’s instructions. Baffled, she looked at the ceiling. Then she looked back at Charcot and said, “It was raining.”
This is what happens in almost every instance of market research. Our buying decisions are like the shoemakers shoes. Theyre usually quite good, but we have little idea how they came into being.
For most of the history of marketing, weve been restrained by the limitations of market research. Its only recently, through advancements in cognitive psychology and brain scanning technologies that were beginning to get a glimpse of what might actually be happening. My next post (tomorrow) why its important that we keep trying.
Comments
Tag: Marketing, Research
Add to 
Del.icio.us | 
Digg | 
Reddit | 
Furl
Have a bookmark! - 
I can never understand what it is that makes a large company think that they are indestructible, but apparently Google continues to believe that because they are the god of search that this will always remain the case. (Disclosure, I work for a search company that was once doing really, really well in search, ahem)
Del.icio.us
Digg
Reddit
Furl
Quote: Universal search has arrived and is here to stay ? This will change how people create content for the web.