Using MaxDiff Analysis To Measure Customer Concerns (Airbnb Case Study)
In their Summer 2025 update, Airbnb introduced a new Services tab for booking on-demand amenities like masseuses, private chefs, and photographers.
While many of these services were already available on Airbnb’s Experiences tab, this update split the two into separate sections for a cleaner user experience. Although that sounds like a no-brainer product launch, it quickly turned chaotic. Social media lit up with Airbnb hosts raising alarms about insurance liability concerns, property damage risks, and rumors that hosts would be penalized for refusing to allow Services in their properties.
A lack of reassurance from Airbnb didn’t help — in 2021, the company shifted from frequent updates to just two big product release events a year, which means that right now Airbnb’s product team is trying to figure out how to improve the Services tab ahead of their Winter 2025 Release event in Q4.
So, what research should Airbnb be running to identify, contextualize, and address hosts’ top concerns about the new Services tab? Here’s one ideal approach…
Collecting host concerns (the easy part!)
Almost immediately after Airbnb Services launched, hosts began voicing concerns en masse. A quick search turns up Reddit and Facebook posts with 100+ comments from stressed hosts, along with YouTube tutorials trying to explain the unclear opt-out process.
Some of the concerns repeated throughout these posts include:
Increased cleaning costs and property damage risks.
Unauthorized commercial activity in residential homes.
Potential insurance and legal problems from accidents.
No opt-out process or refusal mechanism offered to hosts.
These concerns could have been uncovered prior to launching Services through some user interviews and this whole frenzy could’ve been avoided. However, in today’s ship-fast mentality, these things do sometimes happen (especially in multi-sided marketplaces where one stakeholder group can easily be underprioritized).
On the bright side, social media is now filled with free feedback for Airbnb researchers (and me) to trawl through!
Creating a [s̶h̶o̶r̶t̶l̶i̶s̶t̶] longlist of host concerns
Looking through these social media posts, I created a list of 35 problem statements that cover hosts’ concerns about Airbnb’s new Services Tab:
To follow the principles of writing good problem statements for customer surveys, each statement must only include one variable.
An example of a statement with two variables would be: ”Services increase wear-and-tear on my home and I’m not reimbursed for damages or extra cleaning.” Split anything with multiple variables up into separate statements → eg. “Services may cause extra cleaning costs that aren’t reimbursed” and “Services increase long-term wear-and-tear on my home.”
Measuring relative importance of user concerns
Nobody can tackle all 35 of these problems in one go; not even a big-budget company like Airbnb. To prioritize the top concerns to address first, the Airbnb product team would need to know which concerns were the most important, influential, or impactful for hosts.
A great way to rank a list of customer problem statements like this is using a survey format called MaxDiff Analysis.
MaxDiff Analysis is a voting method that breaks a long list of options into a series of smaller voting sets. The survey shows 4 statements at a time and participants are asked to identify the most concerning and least concerning options from that set.
Turning maxdiff survey votes into a ranked list
Thankfully, maxdiff surveys don’t require messy spreadsheets or complex formulas. With a survey platform like OpinionX (which offers free maxdiff analysis surveys with unlimited participants), everyone’s votes are automatically analyzed and the full list of options is ranked for you:
As you can see in the example above, MaxDiff Analysis survey results are easy to interpret, with right = higher concern, left = lower concern.
Each statement is given a score ranging between +100 and -100, where +100 means it was picked as the ‘most concerning’ choice every time, and -100 means it was voted the ‘least concerning’ statement in all the voting sets it appeared in. Maxdiff results can be viewed in a bar chart or a simple data table, like in the GIF below.
Comparing how top concerns vary by host type
Airbnb has many types of hosts; some rent individual rooms in their own home, others rent full houses, and some are professional hosts who manage many properties for different owners. Each of these host types likely have a completely different set of concerns about Airbnb Services.
Filtering the maxdiff results to include only one type of host helps me understand their specific concerns and what’s most important to them.
To enable this filter, just click a bar on any of the Multiple Choice charts and the survey will recalculate the results for that one group of participants. Filtering like this is a great way to see how one group of participants voted, but an even better way to spot outlier opinions is to use a crosstab table.
A crosstab table uses the same starting point as the simple data table; each row shows a ranking statement and its maxdiff score, but it also adds extra columns with scores for different participant groups:
On the table above, the three extra columns with blue numbers show the maxdiff results for three types of Airbnb host: entire home, private room, and specialty rentals.
Looking at the scores for the statement “I don’t want neighbours to see services run from my home” on the 8th row, we see a big difference between the different host types. Those who rent their entire home or specialty accommodation aren’t concerned much by this statement, but live-in hosts renting a private room in their house care a LOT about this — it ranks as their second-highest concern overall:
Comparing how concerns vary by customer segment is the key to building a targeted strategy to address customer frustration after a rocky product launch — that’s exactly why a powerful combo like maxdiff and segmentation analysis should be in every product researcher’s toolkit!
Reset your comfort zone expectations
Many product researchers incorrectly assume that maxdiff analysis is a complicated survey method. I hope this guide shows this is very much not the case. Any researcher, regardless of experience, is capable of running maxdiff surveys to measure people’s needs, preferences, and priorities.
OpinionX is a survey platform for strategic product research. It helps thousands of product teams use powerful survey methods like maxdiff analysis to measure user needs, map customer segments, and model pricing decisions — all with built-in automations to save you from messy manual spreadsheet work. Give it a try!
— — —
Did you enjoy this case study? Thousands of Product Managers, UX Researchers, and Startup Founders subscribe to our newsletter The Full-Stack Researcher for actionable user research advice delivered straight to their inbox:
— — —
About The Author:
Daniel Kyne is the Founder & CEO of OpinionX, the survey platform for strategic product research. Thousands of researchers use OpinionX to measure what matters most to their most important customer segments — enabling advanced research like maxdiff surveys and segmentation analysis on an easy-to-use platform.
→ Try OpinionX for free today