How To Validate New Product Ideas (A Data-Driven Guide)

Validating a product idea isn’t really about your idea at all. It’s about finding a high-priority problem worth solving. Product teams get this wrong in three common ways: asking “what do you think?” (people are too polite to be useful), leading with the idea instead of the problem, and targeting everybody instead of one segment.

The fix is a three-step process, the Discovery Sandwich: discover the problems your target customers face through open interviews, stack rank those problems with a survey so the high-priority ones rise to the top, then deep-dive the winning problem to understand it fully. The best problems to solve are expensive, frequent or boring.

Most articles bore you to death with 5,000 words on why you should validate a product idea before building. This guide skips that and goes straight to the how.

Idea validation doesn't change much by product category, whether you're a product manager shaping a roadmap, a UX researcher testing a new feature direction, or a founder launching something new. It's the same ingredients every time. This guide covers three: the most common mistakes, what makes an idea worth pursuing, and idea validation in three steps.

Step What you do Method
1. DiscoverFind all the problems your target customers experience in your area of focusOpen discovery interviews (no pitching)
2. Stack rankGet customers to collectively rank those problems by priorityA Pair Ranking survey
3. Deep diveTake the highest-ranking solvable problem and learn everything about itTargeted follow-up interviews

The most common idea validation mistakes (everybody makes them)

Mistake 1: "So… what do you think?"

If your approach so far has been to tell people your idea and ask what they think, trash it and start over. Around 90% of people will find something nice to say in these situations, even if they think the idea is terrible.

Instead of pitching the idea to anyone who'll listen, find target customers and get them to talk about their lives: the problems, goals and frustrations they experience. A good place to start is the first few chapters of The Mom Test by Rob Fitzpatrick.

Mistake 2: Idea before problem

Customers don't search for ideas; they search for a fix to their highest-priority problems. They have more problems every day than time, money or energy. For a team to win adoption, the product needs to solve one of the problems at the very top of the customer's priority list.

Shreyas Doshi, a former product leader at Stripe, says this is the big undiagnosed reason so many new products fail: teams validate that their idea solves a problem, but never learn whether it solves a high-priority problem or just a mild inconvenience.

Mistake 3: Target customer = everybody

Your idea launches into an extremely noisy market where nobody knows who you are or what you do. The best way to stand out is to articulate the customer's high-priority problem so clearly that they instantly know you understand what they're trying to solve.

The more customer segments you target, the vaguer your description of the problem becomes as you try to make the messaging relevant to everyone. It's worth reading how WeatherBill burned $16M before realising that hedging its bets on ten different customer segments was killing its potential (it went on to exit for $930M four years later).


What makes a problem worth solving?

You're hopefully already realising that we're not really trying to "validate an idea" at all. We're trying to discover a high-priority problem the team can solve.

But what makes a problem high-priority? The best ones tend to fall into one of these categories, or, better still, overlap two or three at once:

  • Expensive: problems that cost more time, money or energy than the customer can afford to give.

  • Frequent: problems that are cheap or quick to fix on their own, but become high-priority when they recur many times a day or week, burying the customer in repetitive work.

  • Boring: problems that are monotonous, bureaucratic or time-consuming, so they get kicked down the road until a deadline forces action.

So how do you discover these problems in the first place?


Idea validation in three steps: the Discovery Sandwich

There are three steps in the search for a high-priority problem. Together they form a framework we call the Discovery Sandwich:

  • Discover: find all the problems that target customers experience in your area of focus.

  • Stack rank: get those customers to collectively rank the problems from highest to lowest priority.

  • Deep dive: take the highest-ranking solvable problem and learn as much as possible about it.

The qualitative interviews are the two slices of bread; the quantitative stack ranking is the filling that tells you which problem is worth the deep dive.

Step 1: Problem discovery

The goal is to learn about all the problems one group of people experiences during a specific task, occasion or objective (whatever topic you're interested in).

You're completely banned from pitching your idea here. This is an informal interview where you ask open questions about your area of interest and let people bring up problems on their own.

Three quick tips on discovery interviews from The Mom Test:

  • Use follow-up questions to turn any vague or hypothetical answers into concrete, recent examples full of rich detail.

  • If they suggest specific features or product ideas, keep asking "why" until you understand what problem those suggestions were meant to solve.

  • Ignore compliments. If you're getting them, it's probably because you're pitching your idea, which you're not allowed to do anyway.

Step 2: Problem stack ranking

Once you've uncovered a batch of problems, you need to separate the high-priority ones from the mild inconveniences. To do that, mimic how people make decisions in real life: comparison.

I'm a big advocate for Pair Ranking, where participants see two problem statements at a time and pick the one that's more relevant, important or urgent to them. Whichever problem statement wins the highest percentage of these pairs ranks as the highest priority.

I built OpinionXfor Pair Ranking because no other tool offered that survey method. If you'd rather not take my word for it, here's a list of alternative survey tools you can try.

There are just two things you need to create a stack ranking survey:

  • Ranking question: include the context of the use case or topic you want participants to consider when voting, for example: "Which problem is more frustrating when it comes to monthly goal setting with your team?"

  • Problem statements: translate the problems mentioned in your interviews into a list of short statements (here's a guide to writing great problem statements). Include problems you could solve as well as ones you couldn't, because the goal is to find the relative importance across every problem your customers are dealing with.

Share your Pair Ranking survey with a sample of target customers. As votes come in, their top priorities rise to the top of the stack-ranked results.

Some optional extras worth including:

  • Collect basic demographic data and use a segmentation filter to calculate separate stack rankings for each subgroup (for example, managers vs juniors).

  • Gather extra problem statements directly from your early participants and add them to the voting list mid-survey. Customers often have insider vocabulary for problems you haven't spotted yet.

  • Once you've found your high-priority problem, you can repeat this process a layer deeper to explore the hierarchy of specific nuances, but first, the last step of the sandwich.

Step 3: Problem deep diving

Now you've got two valuable data points for the final step: the stack ranked data tells you which high-priority problem to focus on, and it shows which individual participants ranked that problem as high-priority for themselves.

You were careful in Step 1 not to bias the interviews by introducing problem statements yourself. Now the stack ranked data lets you jump straight to the big questions:

  • When was the last time this problem happened for you? Can you tell me the story behind it?

  • How did you deal with it then? What did you use to fix it?

  • What downstream or knock-on effects does this problem create?

  • Why haven't you been able to solve it before now?

  • What does this problem cost you in time, energy and money?

A large dose of curiosity and empathy in these interviews brings out the rich detail you'll use to design every part of your response, from the product itself to your marketing, sales and growth.


An example of idea validation with a stack ranking survey

I once spent 7 months and 150+ interviews trying to figure out what problem our product was meant to solve. Within 2 hours of launching a stack ranking survey, I could see that the problem I'd been chasing was dead last for our target customers: 45th out of 45 problems (here's the full story).

Since that first experience of the power of problem stack ranking, I've seen thousands of stack ranking surveys built by all kinds of teams. To help you get started, there's a sample "Idea Validation" stack ranking survey on OpinionX: create a free account and open it in the Sample Survey Gallery. You can also see how product teams run this end to end on our idea validation use-case page.


Frequently asked questions

How do you validate a product idea? You don't validate the idea directly, you validate the problem behind it. Discover the problems your target customers face through open interviews, stack rank those problems with a survey to find the high-priority ones, then deep-dive the winning problem to understand it fully. That's the Discovery Sandwich.

What's the biggest mistake in idea validation? Telling people your idea and asking what they think. Around 90% of people will say something nice to be polite, so you learn nothing. Lead with the customer's problems instead, and never pitch your idea during a discovery interview.

What makes a problem worth solving? High-priority problems, the ones customers actively try to fix. They tend to be expensive (costing real time, money or energy), frequent (recurring often enough to drain the customer), or boring (monotonous work that gets put off until a deadline forces it).

What is the Discovery Sandwich? A three-step framework for finding high-priority problems: qualitative discovery interviews, a quantitative problem stack ranking survey, then a qualitative deep dive into the top-ranked problem. The two qualitative slices sandwich the quantitative filling.

Previous
Previous

Points Allocation Survey Method [Explanation & Real Examples]

Next
Next

7 Types of Ranking Exercises For Focus Groups (Online & Offline)