What is Discrete Choice Analysis [Models, Tools, Examples]
Discrete choice analysis is one of the most powerful survey techniques available to researchers today — but it’s also one of the least known too. Despite companies like Google, Amazon and Facebook using these techniques to power their user research and market research efforts, discrete choice analysis is poorly understood and often forgotten about.
However, this is also a huge advantage to those of us who do know about it. In this blog post, I cover everything you need to know about discrete choice analysis so that you can start using it to power your own research efforts:
What is discrete choice research?
Discrete choice analysis is a category of research methods that measure the relative importance of a set of options according to a group of people. It shows people multiple options that they have to compare and pick between, thereby measuring which ones they like most by tracking their decisions across a series of votes.
The name “discrete choice modeling” actually explains what it is pretty well! “Discrete” means we’re talking about a bunch of separate things, “choice” means we’re seeing how people make decisions between those things, and “modeling” means using the outcome of those ‘discrete choices’ to predict what people care about most.
People often misunderstand one thing though — discrete choice methods include a whole bunch of different survey formats. Often, people think it only means conjoint analysis (one of the methods I’ll explain in this post), but that is not the case!
What should I use discrete choice modeling for?
Discrete choice modeling is great at simulating how people make decisions in real life — by looking at their available options, mentally comparing the pros/cons of each, and picking the one they like most.
Where it really shines, though, is in scenarios where you’re trying to rank things that are otherwise difficult to measure. Sometimes, things come with objective measurements that make them easy to rank, like movie ratings or box office figures. But lots of stuff comes without pre-existing scores, like which product best solves your problem, or even more so, which of your problems is the most important to solve first.
By using choice-based comparisons (putting things head-to-head and forcing people to pick their preference), discrete choice models can calculate the likelihood an option will be picked by someone in any future comparison.
Discrete choice models are ideal for ranking things based on:
🏆 Preference — what do they like most?
🤕 Pain — what’s their biggest unmet need?
💰 Value — what’s worth most to them?
🚧 Friction — what’s their biggest barrier to action?
❤️🔥 Motivation — what’s their biggest driver of action?
⚠️ Risk — what concerns them most?
Which makes it perfect for research scenarios like:
Customer Segmentation — finding groups of customers that care about the same things.
Roadmap Prioritization — planning what to build by understanding your customers’ top needs.
Assumption Testing — proving whether your hypotheses about people’s preferences are true.
Value Analysis — figuring out which products or features customers perceive as highest value.
Message Testing — seeing which phrases resonate strongest with your target customers.
What research methods are considered discrete choice models?
There are a bunch of different discrete choice survey formats you can pick from for this research:
Ranked Choice Voting — put the list of options in order based on your personal preferences.
Points Allocation — distribute a pool of points according to your personal preferences.
Pairwise Comparison — shows two options at a time in a series of head-to-head votes.
MaxDiff Analysis — pick the best and worst options from a list of 3-6 choices.
Conjoint Analysis — pick the best ‘profile’ (made up of a series of category-based variables) to identify the most important category and the ranked order of variables within each category.
Let’s look at each of these five research formats in closer detail, starting with the one that’s most simple and easy-to-use…
1. Ranked Choice Voting
The best-known format, Ranked Choice Voting shows each respondent the full list of options and asks them to rank them according to their personal preferences. It’s only suitable for lists of 6-10 options max, after which it’s unmanageable for participants (especially on mobile) and incentivizes junk voting. Ranked Choice Voting is a free question type available on OpinionX surveys under the name “Order Rank”.
2. Points Allocation
Points Allocation surveys give participants a pool of points to spend amongst a set of options. The way they allocate those points tells you not only which options are most important to them, but also how much more important those top options are to them. “Points Rank” is a free question type available on OpinionX.
3. Pairwise Comparison
Pairwise Comparison uses a series of head-to-head “pair votes” to compare and rank a list of options. Voting on only two options at a time makes it quick and easy for participants to share their preferences, especially when you’ve got a long list of options to rank. Pairwise comparison uses “win rate” (the percentage of pairs comparisons the option won) as the metric for ranking options. “Pair Rank” is a free question type available on OpinionX.
4. MaxDiff Analysis
MaxDiff Analysis shows respondents a list of options (usually 4-7 at a time) and asks them to identify the “best” and “worst” options from the list. Each time the respondent has finished, the list resets and a new set of options are shown. It works in a similar way to pairwise comparison but collects data faster by requiring some extra cognitive work from each respondent. Maxdiff analysis tends to cost a lot more than Pairwise Comparison though, starting around $4000-5000 for the most popular maxdiff tools compared to the free pairwise comparison tools that are available.
5. Conjoint Analysis
Conjoint analysis measures which attributes are most important to customers when purchasing a product. It presents a participant with 3-6 product “profiles”, each made up of several “attributes”, for them to compare and pick the one they like most. Then a new set of profiles are loaded where the attribute categories are still the same but the attribute “levels” (ie. variables) on each profile have been changed. After the participant has voted a couple of times, conjoint analysis calculates the relative importance range for the attributes, as well as a range for the levels within each attribute.
Conjoint analysis tools are the most expensive of the five methods listed in this post and also require expert consultation in almost all cases to design, administer, and analyze. There are 13 different types of conjoint analysis, each of which is suited to a different research scenario. It is an advanced research method that requires an understanding of data science to effectively manage.
What are some examples of real-life discrete choice experiments?
i. Customer Problem Stack Ranking
The reason I’m writing this post in the first place is to spread the word about discrete-choice research methods after seeing first-hand the value they offer.
Back in early 2021, 6-months after launching OpinionX, we lost our only paying customer at OpinionX. We had interviewed 150+ people trying to figure out what problem our startup should solve but clearly weren’t making progress. We decided to run a discrete choice experiment — we wrote a list of 45 problem statements and sent them to 500 target customers that we found in Slack communities as a pairwise comparison survey.
Within 2 hours, we could see that the problem statement we had built our whole website and product around was ranked dead last as the least important problem! But, luckily, 5 of the top 7 problems were actually things that our MVP could do, so we quickly pivoted everything to focus on these high-priority pain points instead. Within a week of this experiment, we had our first 5 paying customers. Check out the full story behind this experiment in this video:
ii. Psychographic Customer Segmentation
Thousands of gyms around the world, from small family studios to national franchises, use Glofox to schedule their classes, manage memberships, track attendance rates, automate payments, and more.
Francisco Ribeiro, a Product Manager at Glofox, was working on a new feature for Glofox during the summer of 2021 and had already conducted a bunch of user interviews to understand the customer need that this new feature would address. But there was a problem; Francisco couldn’t spot a clear pattern in the needs that customers were talking about during these interviews.
In one interview, a customer would complain about not being able to track engagement with their members and then the next interviewee would say that they have no problem tracking engagement, but that their main challenge was knowing whether members were churning or not.
When we first talked to Francisco, he was taking a step back and had recognized that he was dealing with some frustrating inconsistencies. He decided to run a discrete choice survey on OpinionX to measure which needs his customers felt strongest about. By the end of that same week, Francisco had figured out the cause of his confusion — the size of the customer’s business determined the highest-ranked problem!
Using OpinionX to get customers to vote on their needs via pairwise comparison, Francisco split participants into different groups based on the size of their gym operations to see the ranked results separately for each customer segment. He then calculated Glofox’s bottom-line financial impact for each segment’s highest-ranked problem to understand which one to focus on solving first. With this information on hand, he was able to inform his roadmap prioritization with real data and could easily explain his rationale to the rest of his team.
OpinionX is a free research tool for running discrete choice surveys. It comes with purpose-built analysis that lets you filter your results and compare how people’s preferences change by customer segment. Thousands of teams from Google, Disney, LinkedIn and more use OpinionX to better understand people’s priorities.
Create a free discrete-choice survey on OpinionX
If you enjoyed this guide, subscribe to our newsletter The Full-Stack Researcher for actionable user research advice that we regularly share with thousands of product managers and founders: