Anyone who conducts customer interviews in order to learn about market-problems or to gather feedback knows they provide tremendous value to the direction of your product. For a lot people making these interviews a priority can be difficult. It's true that it takes time to schedule appointments and it can sometimes be awkward when customers think you're just trying to sell them something but I believe there’s another reason holding people back. Hidden behind all of our other anxieties of talking with strangers, I suspect we fear being asked by someone, "how many interviews did you conduct?"
Stakeholders and others on your team may view your findings with skepticism because they have a tendency to depend on quantitative measures such as survey data to inform their decisions. Although surveys and other quantifiable methods have their proper use, others tend to place more significance on quantitative research over qualitative research. This may simply be because it's what they are accustom to and have been conditioned to look for. It could also have something to do with the idea that numbers and charts look like a new business suit —people tend to take them more seriously. It's easy to gravitate toward numbers because we are quicker to associate them with facts and facts are what we want most when forming our decisions.
Being asked how many customer interviews you had can quickly illicit some doubt on your end, but it shouldn't. Just because we can't say that our customer interviews have "statistical significance” isn’t reason enough that we should give up on the practice. Any seasoned product manager will tell you that you’ll begin hearing the same themes or problems repeated somewhere between 5-10 interviews. How then, are you supposed to explain to others that only a handful of conversations with the market place provide enough value to inform your decisions?
In his book Persuadable: How Great Leaders Change Their Minds to Change the World, Al Pittampalli refers to an economic term known as "diminishing marginal returns," which is to say "the incremental value of more information declines over time."
"The same effect holds true when gathering advice for an upcoming decision you need to make. Perhaps you begin by reaching out to a few people. They offer you valuable opinions and insights that you hadn't considered before. As a result, the quality of your decision is likely to go up. But at some point, for every new person you ask, the information is likely to be similar to a previous opinion. You're no longer receiving a lot of new insights. And so the quality of your decision probably won't go up by nearly as much. As you continue to solicit more opinions, the marginal returns for each new opinion begin to get smaller and smaller. At some point, the gains from the new information will even become trivial."
The value of your learnings from customer interviews isn't numeric. The feedback and stories will give you context and meaning to the numbers you'll later quantify with methods like surveys and polls. The point is not to reach statistical relevance with your interviews but to empathize with your customers’ feelings and experiences. Your time with customers is a window for your team to see the impact they are or are not having on the marketplace with your product. The interviews are a pathway to discover new possibilities that you or your team will never see inside your daily bubble.
For example, I asked several from my team to take a look at this piece before I first posted it to see what kind of feedback they might offer. After speaking with the first 5 people, they all mentioned giving some kind of example to help prove my point. Ironically enough, they provided the example themselves and it's the reason for me including this very paragraph. I could continue asking others for feedback, but it has become pretty clear that I either needed to accept what they were suggesting or change up my questions to illicit feedback about more specific things related to this piece. Now that this post has an example, I could reach out to my team again for more feedback or I can move on to validating this post by either surveying people who have read it or just accept the "likes" or "shares" as my quantifying method.
Don't shy away from interviewing your customers because you're not sure if you've done enough. Let your gut guide you on this one. It'll be ok. Just remember the next time someone asks how many interviews you conducted about a certain decision, respond by telling them you began to see diminishing marginal returns after "n" interviews which is an indicator to take the next step.
After the Interviews
1. Foundation for Surveys - Once you've gathered the stories, look for the market-problems you want to validate from those interviews and begin crafting your surveys to prove or disprove what you learned. It's good to get help on these to prevent as much bias as you can.
2. Empathy Generator - As your team begins solving user problems through UX and interaction designs, share the stories you've learned. They will give more life to the personas the team is using to design from. They'll reach a whole new level of compassion for your user. You'll also find that your team will be even more inspired to solve the problem.
3. Give Meaning to the Numbers - When and if you share survey data with your stakeholders or team, liven things up by sharing the stories that influenced some of the survey questions. This will not only inspire your leaders, it will help them embrace the problem. Rather than remembering how big a problem is, they'll leave recognizing it's a problem worth solving.
Everyone loves a good story, and your customers need you to tell theirs to your team.