Comparing The 9 Best Survey Tools For MaxDiff Analysis
Can’t seem to figure out which tool is right for your MaxDiff Analysis survey? I tested the 9 most popular maxdiff survey platforms to see which have the best free versions, how their survey design looks, if they are statistically robust, and whether they offer the analysis features required for running maxdiff research.
Wait, what is MaxDiff Analysis?
MaxDiff Analysis measures people’s priorities by showing them a set of 3-6 options and asking them to choose the best and worst ones. Each time the respondent votes, a new set of statements are randomly selected from the full ranking list. By adding up everyone’s votes, you can rank the options from highest to lowest relative importance.
If you’d like to dive deeper into all things maxdiff, check out our Ultimate Guide to MaxDiff Analysis, where we have details on formulas, setup calculators, and loads of examples.
Here’s an interactive example of a maxdiff analysis survey (may not load for mobile users, but scroll down for a GIF version).
⬆️ interactive example of a MaxDiff Analysis survey built on OpinionX
— — —
What Are The Best Survey Tools For MaxDiff Analysis?
I tested 9 of the most popular maxdiff survey platforms so that you can find the right tool for your research:
Lets look at each individually in more detail…
1. OpinionX 🏆
OpinionX offers advanced market research surveys in an easy-to-use online platform, which is particularly popular amongst product teams and user researchers working to measure user needs, map customer segments, and test pricing changes.
MaxDiff Analysis is one of OpinionX’s most popular ranking methods for measuring people’s preferences and priorities, helping researchers rank any list of option, whether that’s 20 images or 200+ text statements.
The GIF below is an example of a maxdiff survey hosted on OpinionX:
Example of a MaxDiff Analysis survey on OpinionX (via VEED Case Study)
Because OpinionX was built specifically for ranking surveys, it has some unique features that make it especially suited to maxdiff analysis research:
Maxdiff is treated like a normal survey question on OpinionX. While ome other tools treat MaxDiff as a special market research method with limitations for how you can include it in a survey, on OpinionX maxdiff analysis behaves just like any other survey question — drag and drop it into any place in your survey, add as many separate maxdiff questions to one as you need, and easily customize things like your survey colors, the number of options per voting set, and the number of voting sets per maxdiff section. There are no limits to maxdiff on OpinionX, so you can add as many voting options, as many options per set, and as many sets per participant as you need. OpinionX gives you maximum flexibility to use maxdiff analysis however you need.
MaxDiff Analysis survey results chart on OpinionX
Maxdiff results calculate automatically on OpinionX. You never need to handle manual spreadsheets, complex formulas, or mind-melting math. Maxdiff is an accessible research method that anyone can use, regardless of experience, qualifications, or background, and automations like those OpinionX uses are an important part of enabling this accessibility.
Filter your results to compare groups on OpinionX. When you use maxdiff analysis to measure and rank people’s preferences, the first thing you wonder after seeing your results is ‘I wonder how these would change if I only include [subgroup] people?’ OpinionX has a range of filtering and segmentation features to make it as easy as possible to compare groups of people to see how they voted differently, ranging from a one-click filter option to more advanced crosstabulation reports.
Filter your maxdiff survey results on OpinionX in one click
OpinionX removes the complexity from maxdiff analysis. There’s no need for complicated black-box statistics, market research jargon (‘maxdiff’ is already a weird enough name), or advanced logic configurations just to get your survey up and running. On OpinionX, you have a built-in calculator to help decide on your maxdiff design variables (options per set, sets per participant), change the default ‘best’ and ‘worst’ voting labels to whatever you need, and use OpinionX’s sample surveys gallery and one-click templates to see first-hand what a well-designed maxdiff project looks like.
The free version lets you test maxdiff on OpinionX without any commitment. OpinionX’s free tier includes all maxdiff analysis setup and analysis capabilities, allowing you to fully test its maxdiff survey functionality and see if it meets your needs. That is a stark difference from most tools on this list which require you to upgrade to an expensive paid plan or put in credit card details just to see what the maxdiff user interface even looks like. If you decide that OpinionX is the right fit for you and you want to upgrade to unlock a larger participant limit, you’ll benefit from the cheapest prices of any survey tool offering maxdiff analysis surveys.
OpinionX is the only platform that offers image-based maxdiff analysis surveys. Import pictures and ask people to pick the best/worst or most/least liked options to get collective input on logo ideas, design mockups, or sketch ideas.
Partial completes captures votes from people who quit mid-survey. OpinionX is the only survey platform that saves every vote in real time, so if people give up part way through your maxdiff survey, you’ll still be able to include their votes in your analysis if you want. Partial completes functionality is particularly useful for product teams running maxdiff research with their own customers, where outreach is a finite resource and every vote is valuable, unlike recruitment panels where you can just pay the people who complete the full survey.
OpinionX has enterprise security, compliance, and collaboration capabilities. If you need a team research platform capable of passing a lengthy and thorough compliance review from your IT/legal team, rest assured that OpinionX is still the platform for you. OpinionX is used by tens of thousands of teams around the world, including researchers at companies like Google, Amazon and Microsoft, as well as national governments and Ivy League academics. The Accelerate tier includes hands-on procurement processes, security functionality like MFA and SAML SSO, and flexibility to accommodate custom requirements.
These are just a handful of the many reasons why OpinionX has become the number one maxdiff analysis survey platform. To see why teams consider OpinionX their go-to solution for advanced market research surveys, create a free account today and start testing your first maxdiff analysis survey — no credit card or sales conversion required!
— — —
2. Forsta ❌
Forsta (the brand that emerged from the mega-merger between Confirmit, FocusVision Decipher, and Dapresy) offers a MaxDiff Analysis survey format as part of their “dynamic questions” range. I turned over every rock on the internet to find (1) a live demo or (b) a rough price range for Forsta but I couldn’t find anything in the end. Seems like you’ll have to go through a full sales qualification conversation if you want to find this out for yourselves (🚩), good luck to the brave souls willing to go through that 🫡
3. Qualtrics ❌
Qualtrics has a MaxDiff question format but it is not available for use without an existing PX or EX premium subscription or if you have a CX license that includes the journey optimizer. Even with that, you’ll still need to make an additional purchase via your Account Executive to unlock MaxDiff on Qualtrics.
Considering a base-level CX license costs $5,000/year for one CoreXM user, plus $1 per respondent, and Conjoint starts at an additional $8,000/year add-on, it would appear that MaxDiff on Qualtrics would be a $10,000 entry price plus respondent fees for a new customer.
I managed to get access to a test version of MaxDiff Analysis on Qualtrics, which shows a dummy survey that is already filled in with sample data (screenshot above). The dummy survey seems to show that the MaxDiff voting sets are fixed and consistent for every participant, which is weird and not so useful for almost any MaxDiff research project.
4. SurveyMonkey ❌
SurveyMonkey added a MaxDiff question type in October 2023 called “Best-Worst Scale”. Their pricing page shows that it is available on all paid plans, but via a friend’s premium account I was shown that it is only available if I upgrade to Team Premier (the most expensive plan) starting from $2700/year ($75/month x 3-seats minimum x 12-month upfront only). Hard to know which is true, as the information about this feature from SurveyMonkey is contradictory and inconsistent.
Unfortunately, after searching high and low (pun intended) for literally a single example of someone even mentioning it online, I could not find a single screenshot of what their “Best-Worst Scale” actually looks like. Based on an obscure Reddit comment I found, it seems like maybe one person used it in December 2023, but that survey has since been closed so I couldn’t take a look for myself.
Honestly it seems suspicious that not a single person has mentioned this on the internet other than the press release announcing its launch — based on SurveyMonkey’s usage stats 4.2 billion questions have been answered since then and somehow nobody has screenshot or posted anything about this??? SurveyMonkey doesn’t even have any screenshots on their own help center article about this question type, they decided to use text tables instead (screenshot above).
So very weird… My guess, based on the original press release, is that this is a feature of their Market Research Enterprise Solutions range, which would definitely cost a lot more than the stated $2700/year price tag. Hopefully someone can confirm to me whether this actually exists on the self-service product and isn’t just some sort of fake-door test from the PM team there?
5. Sawtooth Software 🟠
Sawtooth Software is a provider of digital market research solutions. Founded in 1983, the company focuses on academics and market research organizations and offers research expertise as an additional available service. The company is best known for
Sawtooth Software is particularly suited to teams looking for deeper technical analysis functionality. It comes with the most advanced statistical testing, making it particularly useful for data scientists and quantitative experts. For example, Sawtooth offers a proprietary version of MaxDiff Analysis which they call “Bandit MaxDiffs” — it leverages Thompson Sampling to algorithmically adapt choice selection based on votes to date for each subsequent participant comparison (this is only available on the Premier tier, which is Sawtooth’s desktop application “Lighthouse Studio”).
Sawtooth Software costs $4500/user/year, with no monthly subscription available. You can create MaxDiff surveys on Sawtooth Software for free but it is limited to 50 respondents. You can also request a demo of their Lighthouse Studio product (a Windows-only desktop application) from their sales team if you want to learn more.
6. QuestionPro ❌
QuestionPro is another of the big survey platforms. It doesn’t meaningfully differ much from products like SurveyMonkey in functionality or price, it’s just another alternative really. QuestionPro offers MaxDiff and Conjoint Analysis questions only in their most expensive tier, which is called “Research Suite” and is available by custom quote only. However, their “Workplace” product starts at $5,000/year, so it is fair to assume that the Research Suite is minimum $10,000/year or more.
7. SurveyKing 🟠
SurveyKing offers a survey platform with a wide range of question types for a low price. Free users can create a test MaxDiff survey, however you can only add three ranking options, so really it’s just a preview of the survey design and can’t be used to actually rank anything. Strangely, there’s nothing preventing the same set of options from appearing many times for the same participant on SurveyKing, which is a red flag for survey usability and data integrity. To unlock MaxDiff for actual survey use, you’ll have to upgrade to the premium version for $19/user/month (which does come with a limited number of responses per month).
8. Alchemer ❌
Alchemer (formerly SurveyGizmo) is a survey platform that, in the company’s own words, “provides tools that rival Qualtrics at a price point closer to SurveyMonkey”. Alchemer offers a MaxDiff Analysis question type but it is only available on their “Full Access” plan (their most expensive tier) starting at $1895/user/year. There is no free trial available and “Full Access” is only for up to a maximum of 3 users — beyond that you’ll have to negotiate a custom enterprise contract.
9. Q Research Software / Displayr ❌
Q Research Software is a data analysis and reporting tool designed specifically for traditional market researchers. The tool is part of Displayr’s portfolio of data products. Q Research offers specialized functionality for this customer segment such as automated data cleaning, formatting, and statistical testing. Unfortunately they don’t have a free tier or even a publicly-available demo of their MaxDiff tool — you must purchase either their Standard License ($2235/year) or Transferable License ($6705/year) before getting your hands on it.
— — —
Conclusion: OpinionX is the best MaxDiff Survey Tool
There is only one tool that offers free MaxDiff Analysis surveys and that is OpinionX. It comes with everything you need to get your first MaxDiff survey up and running — from the built-in calculator to make sure you get a robust sample size to the advanced segmentation for analyzing your maxdiff results. Try it out today, for free, for your next ranking exercise, user research project, or prioritization sprint:
— — —
Not ready to start your MaxDiff survey?
Learn more about MaxDiff in our Ultimate Guide to MaxDiff Analysis, which is full of useful info like real examples of MaxDiff research projects, interactive demos, formulas for calculating ideal sample size, and more.
