Alternatives To MaxDiff Analysis (Tools, Methods, Formulas)
MaxDiff Analysis measures people’s priorities by asking them multiple times to choose the best and worst option from sets of 3-6 statements. Each time the respondent finishes, a new set of statements from the full ranking list is shown.
Most researchers consider MaxDiff Analysis to be an advanced survey format. Unfortunately, so do research software providers, who almost universally include access to MaxDiff formats in their most expensive pricing plans. As a result, many researchers are forced to search for alternative formats that better fit their budget, toolkit or research requirements.
That’s where this blog post comes in! MaxDiff is just one approach to “Best-Worst Scaling” — ie. plotting a set of options from highest to lowest preference using respondent voting. I’ll walk you through 4 other Best-Worst formats you can use instead of MaxDiff and some considerations to keep in mind for each one. Let’s dive in…
^ An example of a typical MaxDiff question with a set of written options and toggles representing Best/Worst on each side.
But First, Say No To Rating Questions
MaxDiff is a comparative ranking method — it forces respondents to compare and choose options according to their personal preferences. This means that MaxDiff creates “continuous” data where answers are all plotted from highest to lowest on a full range of scores. On the other hand, rating questions (eg. rate a statement from 1-5 stars) is “discrete”, meaning that responses are pooled around fixed numbers (like 1 star, 2 stars, 3 stars, etc).
^ This is exactly what we want to avoid. The forced comparison nature of ranking questions allow us to understand even minor differences in people’s preferences.
The gaps between people’s preferences are actually the most important output of our ranked results. If we use rating questions, we lose all this rich data — people’s top priorities all get lumped in together under the 5/5 response. To avoid this, we want to make sure we’re using a research format that forces participants to compare options against each other.
— — —
4 Comparison-Based Alternatives To MaxDiff Analysis Surveys
There are four alternative methods to MaxDiff Analysis that use comparison-based approaches:
Pairwise Comparison
Rank Ordering
Points Allocation
Conjoint Analysis
Alternative 1: Pairwise Comparison
Pairwise Comparison ranks a list of options by comparing them in head-to-head pair votes. By analyzing the number of pairs that a ranking option “wins”, you can measure people’s preferences from best to worst option. Pairwise Comparison works almost identically to MaxDiff — every pair results in a “best” (the winner) and “worst” (the unselected loser) result. Best of all, Pairwise Comparison is a more widely available research method. For example, you can create a free Pairwise Comparison on OpinionX with unlimited ranking options (including image ranking).
^ Example of Pairwise Comparison voting and results on an OpinionX survey
Alternative 2: Rank Ordering
Rank Order questions show respondents the full ranking list and ask them to place them in order according to their personal preferences. It’s the most simple format of the four alternatives explained in this post, but it has some shortcomings worth noting. All the main research providers (Qualtrics, SurveyMonkey, etc.) recommend limiting the number of statements in a Rank Order question to 6-10 max. Beyond that number, you should move your question to Pairwise Comparison, which is a better format for ranking long lists.
^ Example of Rank Order voting and results on an OpinionX survey
Alternative 3: Points Allocation
One disadvantage of both MaxDiff and Pairwise Comparison is that they estimate the preferences of items relative to each other but don’t tell us if our list of ranking options is a good or bad batch from an absolute perspective. That’s where Points Allocation comes in. It gives each participant a pool of credits they can allocate amongst options in whatever way best represents their preferences. The great thing about Points Allocation is it doesn’t just show the relative preference, it shows the magnitude of their preference — for example, we don’t just learn that Simon prefers apples to bananas, we see that he would give 9 of his 10 points to apples.
^ Example of Points Allocation voting and results on an OpinionX survey
Alternative 4: Conjoint Analysis
Conjoint Analysis is used to understand the influence that individual aspects/features of a product or offering exert on the buyer’s decision. It’s more advanced computationally than MaxDiff and used for much more specific use cases, typically in bundling analysis and pricing/packaging research. Conjoint Analysis is one of the most expensive survey formats out there, so expect to pay thousands for any platform (Conjointly’s pricing plans start at $1,795 per user per year with no monthly options).
^ Example of a Conjoint Analysis question
Create A Free Ranking Survey Today
There’s no reason to be stuck on only using MaxDiff when similar methods are more widely available. Thousands of companies including Google, Amazon, and Shopify use OpinionX to run their ranking and segmentation research projects. Create a free ranking survey using Pairwise Comparison, Rank Ordering or Points Allocation on OpinionX today!
Not ready to jump into creating a survey just yet?
Not to worry, we’ve got a range of guides that you’ll want to check out:
