Discrete Choice Experiment: Methods, Examples, Tools
“A discrete choice experiment shows people a set of options and asks them to pick one, then works backwards from thousands of those picks to estimate what they value. Conjoint analysis is one type, not the whole category. This guide covers what a choice set looks like, five formats, and how to pick between them.”
Large technology companies run these for user research and pricing studies. Outside market research, they get very little attention.
Contents
What is a discrete choice experiment?
What are they used for?
What does a choice set look like?
The five formats
Which format should you choose?
Worked examples
What is a discrete choice experiment?
A discrete choice experiment is any survey method that asks people to compare a set of options and make decisions based on their personal preferences and priorities. Analyse enough of those individual decisions and you can model how people choose and what matters most to them.
Plenty of survey types sit under the umbrella, from simple head-to-head voting on two options at a time to formats like conjoint analysis, where participants choose between profiles built from combinations of attributes such as price, colour or size.
The name explains the mechanism. Lists of distinct options (discrete) that participants pick from (choice), run under controlled conditions to estimate real-world behaviour (experiment).
Conjoint analysis isn't a synonym for the category. Conjoint is one type. This guide covers five.
None of this is new. Daniel McFadden took the 2000 Nobel Prize in Economics for the theory and methods behind analysing discrete choice, and the random utility model he formalised is what converts a pile of picks into a number you can defend.
What are they used for?
Researchers measuring preferences often reach for a rating scale:
"Rate your satisfaction with ____ from 1 to 5 stars."
The trouble is that nothing stops a participant awarding four stars to all twelve items, and plenty of them do exactly that. What comes back is a list where everything scored roughly the same, which tells you nothing about what anyone would give up.
Two named failure modes cover most of it: central tendency bias, where everything drifts to the middle, and straightlining, where people stop reading and pick the same column all the way down.
Choice-based methods close that escape route. Two or more options sit side by side and one has to be chosen, which is how decisions work outside a survey. You don't get to keep everything.
When should you use one?
The format earns its place when what you're ranking has no number attached to it already.
Movies have Rotten Tomatoes scores and songs have stream counts, so sorting those is trivial. The questions that matter rarely come with a score:
What product features do users care about most?
What frustrations are most painful for your customers?
What benefits do clients seek most from our services?
Nothing about those has an objective answer waiting to be looked up. A discrete choice experiment builds one out of decisions instead.
The six lenses
Preference. Which do they actually like?
Pain. Which problem hurts most?
Value. Which feature is worth paying for?
Risk. What are they worried about?
Friction. What's stopping them?
Motivation. What would make them move?
Six research projects that use them
Customer segmentation. Which customer types feel which pain hardest, which is the input for needs-based segmentation.
Roadmap prioritisation. What to build next, decided by users' unmet needs instead of by whoever argued hardest.
Assumption testing. Putting a team hypothesis in front of customers and finding out.
Value analysis. Which parts of the product users actually rate, as opposed to which parts took longest to build.
Message testing. Which framing lands with the audience you're aiming at.
Pricing studies. What people will pay for, and the number they'll pay.
What does a choice set look like?
This is the part most guides skip. A discrete choice experiment presents choice sets: small groups of options shown together, where the participant picks one.
Take the smartphone example. First you define your attributes and the levels inside each one:
| Attribute | Levels |
|---|---|
| Brand | iPhone, Samsung, Google Pixel |
| Storage | 128GB, 256GB, 512GB |
| Battery life | 1 day, 2 days, 3 days |
| Price | $599, $899, $1,199 |
Then the survey builds profiles by taking one level from each attribute, and shows two or more of them together. That's a choice set:
| Profile A | Profile B | |
|---|---|---|
| Brand | Samsung | iPhone |
| Storage | 512GB | 128GB |
| Battery life | 1 day | 3 days |
| Price | $899 | $1,199 |
Pick one. Then another set appears with different combinations, and you pick again.
Nobody ever says "storage matters more to me than battery life". They just keep choosing, and after enough choice sets what they surrendered each time shows you what they wouldn't. That's the whole trick.
Which survey formats count?
Five formats cover almost every use.
Method 1: Conjoint analysis
Conjoint fits when the options you're ranking have multiple layers of information, like the smartphone example above.
It answers two questions at once:
Which attributes influence people's decisions the most?
Which specific levels within each attribute are preferred most?
How does conjoint analysis work?
Each round puts 2 to 5 profiles in front of the participant, every profile carrying one level from each attribute. They choose one. Repeat that a handful of times and the data supports an estimate of which attributes carried the decision and which levels within them did the work.
The setup is less intimidating than the output suggests. Below is the survey that produced the example above, which amounts to attribute names with their levels typed underneath.
Four places to go deeper: what conjoint analysis is, how the results get calculated, what the tools cost, and when the method is the wrong choice.
Conjoint tooling is expensive across the market, and the full tools breakdown carries the current figures. On OpinionX, conjoint sits on the free tier alongside every other method, capped at 25 participants per survey. Paid plans start at $900 a year and remove the cap.
Method 2: MaxDiff analysis
MaxDiff shows 3 to 6 options at a time and asks participants to identify the best and worst in each set. Aggregating the responses ranks every option from highest to lowest preference.
The best and worst labels can be changed to whatever suits: Most and Least Liked, Top and Bottom Priority, Love and Hate.
Long lists become manageable because nobody ever sees the whole list at once. Product teams reach for it when customers need a vote on features or problem statements.
It's straightforward to run and carries a reputation as an advanced method, so it tends to be priced that way. Our comparison of MaxDiff tools covers what the alternatives charge and which let you test before buying.
Method 3: Pairwise comparison
Pairwise comparison is the simplest of the five. Participants compare two options and pick one.
One decision, two seconds, repeat. Because each screen asks so little, the list behind it can run to hundreds without the task getting harder. It's the right choice wherever a drag-and-drop question or a rating grid would exhaust people.
Scoring is a win rate, meaning the share of its pairings each option took. Bradley-Terry, published 1952, is the formal model sitting under it.
Method 4: Ranked Choice Voting
The best-known format for ranking preferences, and the one most people have met through ranked-choice elections. Participants order the full list by personal preference.
Cap it at 10 options, and 3 to 6 is better. Ordering a longer list in one go is hard on a desktop and worse on a phone, and what you get back is junk.
On OpinionX it's the "Order Rank" question type.
Method 5: Points allocation
Points allocation gives each participant a pool of points to distribute across the list however they want. It suits budget allocation studies and any situation where you're exploring trade-offs between priorities.
Ranked choice voting tells you the sequence. Points allocation tells you how far apart the items sit.
Ranking three ice cream flavours puts Chocolate first, Vanilla second and Strawberry third, and stops there. Handing out 10 points and watching 9 of them land on Chocolate says something the sequence can't: the other two barely register.
"Points Rank" is available on OpinionX with a custom number of points and as many options as you want.
Which format should you choose?
Three categories, five formats.
Full list ranking. You see the whole list at once. Ranked choice voting and points allocation.
Sampling exercises. You vote on a series of small sets that add up to an overall result. MaxDiff and pairwise comparison.
Multi-variable scenarios. You're ranking a list of options that each contain their own range of options. Conjoint only.
Full list ranking
Short lists suit both. Pick points allocation if you need to know how strongly people feel, ranked choice voting if the sequence alone answers your question.
Sampling exercises
Past ten options, it's MaxDiff or pairwise, and honestly either will do.
The tiebreakers: wordy or technical options favour pairwise, because two options are easier to hold in your head than six. A small participant pool favours MaxDiff, because each screen extracts more information.
Multi-variable scenarios
Only conjoint handles a list where each entry contains its own list.
Software pricing plans are the everyday case. Every plan carries the same features at different allowances: surveys per month at 1, 10 or unlimited, researcher seats at 1, 3, 5 or unlimited, participants at 100, 500 or unlimited. Ranking the categories and the allowances inside them takes one conjoint survey and can't be done any other way.
Worked examples
Customer problem stack ranking
This guide comes out of what these methods did for us directly.
Early 2021, six months in, and our only paying customer left. 150+ interviews behind us and no clearer on what we should be solving.
So we wrote out 45 problem statements, put them into a pairwise survey, and sent it to target customers we'd found in Slack communities.
Two hours later the answer arrived: the problem statement the entire website and product had been built around came last. Five of the top seven were problems the MVP already handled, so everything pivoted onto those. Paying customers followed within the week.
Customer Problem Stack Ranking carries the whole story.
Psychographic customer segmentation at Glofox
Thousands of gyms worldwide use Glofox to schedule classes, manage memberships, track attendance and automate payments, from single family studios up to national franchises.
In summer 2021, Glofox product manager Francisco Ribeiro had a new feature in progress and a stack of user interviews meant to justify it. What he didn't have was a pattern.
Interview one: we can't track member engagement. Interview two: engagement tracking is fine, our problem is spotting churn before it happens.
Francisco ran a choice-based survey to measure which needs his customers felt most strongly about. By the end of that week he'd found the cause of the confusion: the size of the customer's business determined the highest-ranked problem.
He'd run the needs through a pairwise survey, then split participants by the size of their gym operation and read each segment's ranking on its own. Attaching a financial impact to each segment's top problem decided the order. What he took to his team was a prioritised roadmap with the reasoning attached.
How Glofox used OpinionX for needs-based segmentation has the detail.
Frequently asked questions
What is a discrete choice experiment in simple terms?
A survey that shows people small sets of options and asks them to pick one, repeatedly, then works backwards from the pattern of picks to estimate what they value. The name describes it: distinct options, a forced choice, run under controlled conditions.
Is conjoint analysis the same thing?
No. Conjoint is one type, the one where each option is a profile built from multiple attributes. MaxDiff, pairwise comparison, ranked choice voting and points allocation all belong to the same family, and none of them are conjoint.
What is a choice set?
The small group of options shown together on one screen, from which the participant picks. A conjoint choice set holds 2 to 5 profiles. A MaxDiff set holds 3 to 6 statements. A pairwise set holds two.
What are attributes and levels?
Attributes are the dimensions of whatever is being tested, like brand, storage and price. Levels are the specific values inside each one, like iPhone, 256GB and $899. A profile takes one level from each attribute.
Why not just ask people directly what they value?
Because stated importance and revealed preference diverge badly. Ask directly and everything scores highly. The method never asks the question, it just records what people gave up, repeatedly, until the pattern becomes clear.
Every format above runs on OpinionX. Surveys, questions and researcher seats are all uncapped; participants stop at 25 per survey on the free tier, which covers a full study end to end. $900 a year removes that limit.
Over 42,000 researchers and product people get one method breakdown like this each week in The Full-Stack Researcher.