So you’re looking to run a survey to rank people’s preferences or priorities but you’re not interested in using MaxDiff Analysis. This blog post will help you find the right solution for your project using a series of rapid-fire questions.

What is MaxDiff Analysis? MaxDiff Analysis is a survey format that measures people’s priorities by asking them multiple times to choose the best and worst option from sets of 3-6 statements. Each time they vote, a new set of statements from the overall ranking list appears. For more info, check out our Ultimate Guide to MaxDiff Analysis.

Are there any free MaxDiff Analysis tools? Yes (just one that I know of). The most common reason people land on this post is because they’ve realized that MaxDiff is an “advanced” research method so companies charge crazy prices for their MaxDiff tools. I tested the 10 most popular MaxDiff tools and only one had a genuinely free version (spoiler here).

Is MaxDiff actually “advanced” or complicated? No. There are advanced versions of MaxDiff Analysis that used regression-based scoring algorithms, but for 99% of projects this is overkill and unnecessary complexity. OpinionX uses a simple formula to calculate MaxDiff scores → (best-worst)/appearances. For example, an option appears in 100 voting sets where it is picked as the “best” 40 times and “worst” 5 times will have a score of (40-5)/100 = +35%.

Ok, thanks for the info but I just want to find a different research method… Can you help? Yes. What makes MaxDiff great is that (1) is created forced comparisons which are great for simulating trade-off scenarios, and (2) they produce Best-Worst Scaled data which ranks a set of options by relative importance even when they don’t have any objectively measurable characteristics. Here are four other methods that also meet these two criteria:

But first, say NO to rating scale questions. Rating questions (like a 1-5 star scale) allow people to pick the same score for every option, which is like the worst case scenario for your research. This is called the central tendency bias and the best way to avoid it is to use a comparative ranking method like MaxDiff or one of the other 4 methods explained below…

— — —

4 Alternatives To MaxDiff Analysis Surveys

There are four alternative methods to MaxDiff Analysis that use comparison-based ranking:

1. Pairwise Comparison

2. Ranked Choice Voting

3. Points Allocation (aka Constant Sum)

4. Conjoint Analysis

Alternative 1: Pairwise Comparison

Pairwise Comparison ranks a list of options by comparing them in series of head-to-head pair votes. By counting the percentage of pairs an option “wins”, you can easily rank people’s preferences from best to worst option. Pairwise Comparison works almost identically to MaxDiff but with only 2 options instead of 3 or more, making the voting process a lot quicker and easier. OpinionX lets you create unlimited pairwise comparison surveys for free with as many participants per survey as you’d like. It also has image-based pairwise comparison too!

— — —

Alternative 2: Ranked Choice Voting

Ranked Choice Voting gives each respondent the full ranking list and asks them to place the options in order according to their personal preferences. It’s the most simple format of the four alternatives explained in this post, but it has some shortcomings worth noting. It is recommended that you limit the number of statements in a Rank Order question to 6-10 max. Beyond that number, you should move your question to MaxDiff or Pairwise Comparison, which are both better formats for ranking long lists. The free tier on OpinionX comes with a free ranked choice voting tool, which you can test in the embedded example below:

— — —

Alternative 3: Points Allocation

One disadvantage of both MaxDiff and Pairwise Comparison is that they estimate the preferences of items relative to each other but don’t tell us if our list of ranking options is a good or bad batch from an absolute perspective. That’s where Points Allocation comes in (also known as a Constant Sum survey). It gives each participant a pool of credits they can allocate amongst options in whatever way best represents their personal preferences. It doesn’t just show the relative preference, it shows the magnitude of their preference. For example, Points Allocation helps us learn more than the fact that Simon prefers apples to bananas — we see that he would give 9 of his 10 points to apples (he really prefers apples). You can create free points-ranking surveys on OpinionX like the example below:

— — —

Alternative 4: Conjoint Analysis

Conjoint Analysis is a multi-factor ranking method that asks participants to vote on profiles that each contain multiple variables. It’s used to figure out how important different aspects/features of a product are in the overall offering for a buyer. For example, Conjoint Analysis will help us rank the importance of a phone’s battery life, storage space, and color, while at the same time helping us rank the various possible colors too (eg. black, white, rose gold).

As you can probably guess from that description, Conjoint Analysis is a lot more complex than MaxDiff Analysis or the other comparative ranking methods explained in this guide. As such, it is also a lot more expensive — see this breakdown of the most popular conjoint analysis tools and their prices (the tools range from $2000 to $30,000 per year).

The good news is that we’re in the process of building a free conjoint analysis question on OpinionX, which will be available in June 2024. Once it’s ready, you’ll be able to see it in action here.

— — —

Create Your Own Free Ranking Survey

There’s no reason to be stuck on only using MaxDiff when similar methods are more widely available. Thousands of companies including Google, Amazon, and Shopify use OpinionX to run their ranking and segmentation research projects. Create a free ranking survey using Pairwise Comparison, Rank Ordering, Points Allocation, or MaxDiff Analysis on OpinionX today!

Not ready to jump into creating a survey just yet?

Not to worry, we’ve got a range of guides that you’ll want to check out: