Using focus groups to answer questions is asking them to perform an unnatural act. Like asking a classroom of pre-schoolers to sit and listen, asking a group of PTA parent volunteers to check their egos at the door, or asking your family to just get along during the holidays, groups of seemingly normal people collapse into warring factions like so many Afghan warlords.
Focus groups don’t make decisions. They are absolutely marvelous at providing insights that you might have missed, asking questions you might have overlooked, and generally fleshing out concepts that you have begun. Never ask them to settle arguments, though, because you’re only asking for trouble.
Why? First, and least interesting, is that the sampling error on “8” is very high. But why take the trouble to post an entry if I’m only going to show you the math? The problem with focus groups is bias. What does this look like? Let’s go to the videotape and review the cast of focus group characters…
Don’t worry — regardless of your industry, your research firm, or your moderator, you will have the exact same group of people in your facility, drinking Sprite and looking self-consciously in the one-way mirror.
First up, we have Alpha Male. Alpha is almost always a guy. Alpha has an opinion. On anything you say. And he’s not shy about expressing it. He will be less shy every minute that passes. And he only addresses comments to the moderator. To Alpha, this is a one on one discussion. The others are just audience.
Next, in all meanings of the word, we have Beta Male. Also, almost always a guy for some reason. Beta will mirror just about everything that Alpha says, moments after Alpha is finished speaking. Beta is a wonderful ‘yes man’. Beta may restate what Alpha has said, just for variety, but will always sheepishly come back to agreeing with his superior.
And in the other corner… wearing the blue trunks… we have The Contrarian. Contrarian has one purpose in life. That purpose is to contradict whatever position Alpha has taken. Contrarian never deems Beta worthy of eye contact. Alpha doesn’t care what Contrarian has to say.
Somewhere indescriminately between these two factions — if you’re lucky, because you don’t always get him — is The Purely Rational Man. R-Man is great because he will stop all discussion in a fairly pedantic way to bring everyone back to first principles — his — and explain — again — how he makes decisions. And, for some reason, it’s usually a ‘he’. The best part about R-Man is that inevitably, by the 45th minute of the focus group, he has painted himself into a corner and can no longer meaningfully contribute. This really makes him sad, confused, and disillusioned. Great fun to watch, of course.
The other three to four participants are generally categorized as Inactive Ingredients. They are there for the $20 bucks. Just leave them alone and they won’t bother you, either. To the tape…
Moderator: “So, group, what color is the sky, usually?”
Alpha: “Jim, the sky is blue. Bluer than hell. It’s always blue. I can’t understand why people don’t just get on with the blue-ness of the sky. So blue it stinks. It should be purple, if they knew what they were doing.”
Beta: “You know, gosh, I just have to agree that the sky is pretty blue! It really is. I’ve given it a lot of thought — a lot, really — and I’ve tried other colors in my mind, but frankly I always come back to that old, darn blue!”
(The Moderator will now visually, with eyebrows arched, encourage the Inactive Ingredients to respond. All will give the moral equivalent of folding their cards and avoiding eye contact. They are waiting out the clock for the twenty).
The Contrarian, when sure they have the floor: “The sky is actually red if you look at it closely and is never blue. Frankly, it’s odd that some people actually think they are seeing blue, when they are actually just seeing red. I’ve headed up task forces to get our best thinking on sky color at work, and we’ve always — always — come up with red. The sky is red.”
Purely Rational Man: “Let’s all take a step back, people. I always say, ‘it’s best to look out the window before making grand pronouncements about what color the sky is.’ If I can’t see it myself (… and here, he looks pointedly at each and every other member of the focus group over his bifocals), then I don’t trust it.”
Moderator: “But it’s dark outside.”
Purely Rational Man: “Yes. Yes, it is. It’s dark right now. Isn’t it.”
(Purely Rational Man is no longer able to keep his body upright and slowly sinks into a boneless puddle on the table. Others move their notepads and Sprites out of the way, but take no other notice).
Alpha: “Blue, Jim. Blue blue blue.”
If you’re really lucky, you’ll get a true once-in-a-career wild card. Mine was the guy with Tourette’s Syndrome.
Forcing iterative work to happen — making people build on others’ ideas — as well as smaller team break-outs can split these archetypes into smaller pieces and often snuff out the effects of bias. After all, you paid a few grand for this party and you might as well get something of value from it.
This same self-selecting role assignment plays out in more than just focus groups — look hard at your next meeting. Assume the senior most person or the person with the agenda is the moderator. See which peers fall into these comfortable roles.
Is this pack mentality? Maybe so. Assuming none of your peers yelps uncontrollably, of course.
Copyright (c) 2006 Stephen Denny
Al Ries writes that marketers should never ask people what they’re GOING to do (they’ll never answer future questions correctly, how can they?)…only what they HAVE done. Couldn’t agree more.
I’ve another friend who espouses the best focus groups are held after everyone has had just 1 good drink (but not 2+), that way they’re honest. You know, it makes a lot of sense and would be the most fun–and productive–focus group ever. Just give them a martini and some finger food and ask ’em what they really think.
I absolutely must do that someday soon. Would make for a great set of results–and a fun article.
Thanks for the good thinking.
CK — thanks for your note! Feel free to send me an invite to your inaugural “two martini qualitative exploration”. Don’t really care what the subject is, either. I’m sure after a few drinks, I’ll have an opinion on whatever we’re talking about.
Focus groups are great for developing new insights, raising new questions, and for understanding how issues affect groups. But they’re lousy at answering questions.
And yet I’d bet that better than half of all groups are done solely for the purpose of validating a preconception the marketer went in with. Where’s the justice??
“And yet I’d bet that better than half of all groups are done solely for the purpose of validating a preconception the marketer went in with. Where’s the justice?”
I’ll bet my savings that better than half is right at 90%. Yep, they’re used for validity, not identification & insight.
And 0% of them include martinis. But do stay tuned :-).