Why user interviews can fail you when starting a start-up

We all know how important it is to conduct interviews on Day 0, but they aren't a magic bullet

So you want to start a start-up and you have a general sense of what ideas you want to go after… what do you do as a product manager when there’s no product to manage?

Age-old advice is to start interviewing a bunch of people - in some cases, there are stories of founders interviewing hundreds of people to hone in on what they are trying to solve. I’ve personally found that taking it too far results in wasted work, repetitive interviews, and definitely worse, incorrect conclusions.

Prior to starting Correlated with my co-founders, I spent almost a year working on various ideas, doing research, and conducting - you guessed it - interviews. The interviews were really helpful in understanding the problems I wanted to work on and who they resonated with. I ultimately decided to stop working on those ideas for a lot of reasons that I’ll (potentially) share in the future. Fast forward more than a year, and Correlated is up and running with multiple customers using our product and employees in the dual digits! At Correlated, we also talked to a bunch of customers (and still do), and here are some take-aways that stand out.

Take-away 1: It’s really hard to avoid bias in user interviews

During my pre-Correlated year, I conducted many interviews across many personas - and I tried my best to run a consistent process throughout. I had a set number of questions that were open-ended and problem-focused. I would do those exploratory questions first before sharing a potential solution I was thinking about and getting feedback. I would grade the feedback I got based on how important it was to the user, and use that numerical number to figure out if I was on the right track or not.

The problem is, there are a lot of ways bias can inject itself into the process:

- The people you initially interview are likely friends of friends, and are unlikely to sh*t on your idea

- Despite starting with open-ended questions, you’ll inevitably follow up with leading questions that validate your own assumptions

- Starting a company is really personal, and it’s hard to separate your emotional attachment to an idea from actual feedback

Which means that you can’t overweigh the findings you get from your user interviews because it’s likely that you’re simply suffering from confirmation bias.

Take-away 2: Users have a lot of trouble envisioning a solution and providing actual feedback. Instead, focus on understanding the problem and developing user empathy.

If you (like I did), try to pitch a solution to a customer, you will almost inevitably NOT get the feedback you want. The reason is that it’s really hard for a customer to envision something completely new. You’ll end up getting lukewarm responses to things that are complex (but ultimately make a lot of sense and are differentiated), and positive responses to obvious solutions to a problem. Rather, I’ve found that it’s more useful to really understand the problem you’re trying to solve and work on that, rather than expecting users to tell you what to build.

That being said, there are cases where providing a solution and getting feedback on it are useful - this is typically when the product exists, you’re working on a specific feature, the problem is well-defined, and you’re talking to a user who would implement the solution themselves and are domain experts.

Take-away 3: Don’t be afraid to pitch an idea during the interview

So this contradicts what I described in Take-away 2, but I think the key is doing 100 interviews with users on the “problem” will only get you so far. At some point, you have to start pitching an idea. Sure, the responses might not be right, and your gut based on developing user empathy might tell you to do something different, but you have to put yourself out there and start getting feedback. However, there are good (and bad) ways to do this, which leads me to…

Take-away 4: Start putting tangible things in front of users as soon as possible

Ok, so I mentioned that pitching an idea to someone won’t get you the feedback you need, but that you should also still pitch ideas anyway. However, I think the key is that the minute you have a general idea of what something might look like, you should get it down on paper. Maybe it’s a mock or a wireframe, or a presentation, but you need to give people tangible things to look at in order to give you real feedback. When I was working on InPlace, which was an application for seniors, the feedback we got from user interviews was very different from the feedback we got once we started putting an example survey in front of people for them to use.

To close out, some quick tips on user interviews and how not to waste your time based on my experiences conducting interviews:

- Once you hit around 20 interviews of the same persona, it’s time to either try a new interview script (like adding in a pitch, a wireframe, etc), or trying a different persona

- Find a friend (preferably your co-founder) to listen in on user interviews so that you can help each other avoid bias in the interview process

- Think of ways to make your product more tangible to your interviewees in the fastest way possible

- In the beginning, focus on understanding your persona and the problem you are trying to solve