4 sample opinion strategies for your stack ranking surveys
Are you suffering from blank page syndrome when you try to create a stack ranking survey?
This post explores the four most common strategies that successful stack rankers use when creating their surveys.
These four strategies are used by product, UX and research teams at top companies like Adidas, Hotjar and Canva to design their stack rank surveys and get the data they need to make better prioritization decisions.
With real-life examples for each strategy, I guarantee that you won't make it to the bottom of this post without finding inspiration for your next OpinionX stack ranking survey.
For questions like how many sample opinions to include in your survey, check out this FAQ blog post for stack ranking newbies.
Let's dive in 💥
Sample Opinion Strategy 1: Problem Statements
Popularized by the popular idea validation method from Stripe called Customer Problem Stack Ranking, using problem statements is the most common way that teams build their stack rank surveys.
Examples of problem statements as sample opinions:
The inability to export data as a .csv.
Not being able to add my teammates as collaborators.
It's frustrating that the platform can't be white-labeled.
The payment process requires a lot of steps.
There's no way to set limits on the dashboard.
I need to get approval from my boss each time before going live.
The best way to create these statements is to first look at the feedback, feature requests, and interview transcripts you've gathered over the past quarter.
Tips on writing good problem statements:
Keep them short and concise.
Each statement should have only one main point.
Don't include personal information like people's names.
Keep the perspective of the statements consistent (ie. write the statements in the first-person perspective).
Try phrases like "Not being able to..." or "It's frustrating that..." if you're finding it hard to start.
Don't forget to include interpersonal problems as well as technical ones — those are often the biggest barriers that people are facing!
Using problem statements as your sample opinions is ideal for a bunch of different project types, like...
Measuring which are the most important problems to prioritize on your product roadmap for the upcoming quarter.
Validating whether your idea solves an important problem for target customers.
Identifying the barriers that prevent users from reaching a 'Desired Outcome' during product discovery, which will point the way to new opportunity spaces (ie. the Teresa Torres "Opportunity Solution Trees" framework for product discovery).
For examples of stack rankers using problem statements for their sample opinions, check out these case studies:
Sample Opinion Strategy 2: Feature Ideas
Most product roadmaps out there are based on features or outcomes (we're personally big fans of problem roadmaps), so product teams tend to be very used to working with feature idea statements.
It's becoming more popular to shy away from using feature ideas in any user engagement because users just love upvoting ideas even if they don't solve the actual problem.
But because OpinionX stack ranking surveys use a voting method called "Pair Choice Voting," it means that participants are forced to consider the trade-offs and benefits of each idea before voting each time.
With a robust process in place to inform feature idea statements (ie. don't just take ideas straight from users, it's important to first understand the problems those users think their ideas are solving), then using feature ideas for your sample opinions can still work really well.
Because stack ranking on OpinionX is based on comparison, the best way to use feature ideas for your sample opinions is to include multiple possible solutions to the same problem.
For the examples below, imagine that a product team at an online invitations platform is considering adding new ways of sharing invites to help boost product distribution.
Examples of feature ideas as sample opinions:
Import people's emails via .csv.
Links for sharing publicly to different social media platforms.
Referral tracking to reward people who forward their invites.
Mail merge feature for sending personalized invited en masse.
Pop-up widget to invite users while they're on my app/website.
Embed code for adding an invite card iframe to my page.
Twibbon creator for adding my invite code to my social media profile picture.
See the way these are all potential solutions to the same problem? This way, participants are comparing all the options from the same problem perspective.
If you're trying to consider solutions to multiple problems, I would recommend going with a Problem Statement approach instead.
Top tip for writing feature ideas as sample opinions:
Push yourself to consider how the feature idea might be misinterpreted. Two different participants could easily come to separate conclusions that your feature idea is to solve completely different problems.
To avoid that scenario, consider testing your survey with your team first to double-check your sample opinions before distributing it to your users/customers.
Sample Opinion Strategy 3: Customer Values
Customer values can be either the motivators that drive their behavior or the aspects of your product/service that they derive the most value from. I'll break this section down into two parts to explore each.
Motivators of Behavior
A great example of a stack ranking project used to understand people's motivations was one of our very first pilots with Irish charity Feed the Heroes.
The team at FTH took a bunch of quotes they had received from donors via email, social media and one-to-one conversions, and added them all as sample opinions on their OpinionX survey. Then, they sent the survey out to their thousands of donors to understand why people were supporting their campaign.
In the end, the FTH team discovered that there one of the driving motivators behind people's donations was something they had completely overlooked. Check out the case study for more info: Feed the Heroes gets closer to their donors core values using OpinionX.
Product Value Drivers
Pricing research is incredibly difficult.
The inability to simply ask users what price they would pay or what features should go in each tier leaves teams experimenting with trial and error approaches and frustrating everybody involved.
By combining OpinionX's segmentation features with a set of customer values as sample opinions, you can quickly understand which features your customers on each tier or lifecycle stage derive most value from.
For example, freemium users might get their value from core shareability features, premium users might see the collaborative features as the most valuable feature to them, while enterprise customers might place customization options as their highest value feature.
Within this example, you'll see that stack ranking product value drivers revolves around your existing feature set.
Examples of customer values as sample opinions:
Adding teammates as collaborators.
Segmenting the final data based on the contract value of the customer.
White-labeling the billing experience.
Dedicated customer support.
SSO sign-in options.
Successful case studies from other customers.
Tips for writing sample opinions to understand product value drivers:
Like with problem statements, don't forget to include statements for the non-feature values that your customers get, such as support agent time or product documentation.
Focus on existing values rather than hypothetical future ones — mixing these will lead to confusion amongst participants as they try to compare them during voting.
When to use the customer value approach for your sample opinions:
To understand which tiers you should split features into.
To identify the value metric that you should use for pricing (ie. seats, participants, items processed, etc.)
To improve the clarity of your product positioning and marketing.
To validate whether a recently shipped feature has gained mindshare amongst customers yet.
Sample Opinion Strategy 4: Variable Testing
Analysis paralysis can easily kill your momentum as you get closer to reaching a project's conclusion.
Selecting between a shortlist of validated variables often leaves product and design teams resorting to hunches, biases and assumptions. After building the entire project around data-informed decisions, the final one is so often made in the absence of any data.
That's because this is the same type of data you'd get from an A/B test, but for an A/B test you have to actually build and ship each option. Sometimes, that simply isn't an option.
That's when teams turn to stack ranking for variable testing. This works similarly to our feature idea example above — your sample opinions are all rooted in the same point, but each has a minor difference.
Examples of times that teams have used OpinionX to stack rank potential variables include:
Picking the ice breaker games for the annual office Christmas party.
Subdomain structure (brand.website.com versus website.com/brand/ versus brand-website.com).
Testing slogans and names for a new product.
Comparing value propositions to see which resonates best with target customers.
Experimenting with titles and copy for a Facebook ads campaign.
Bonus Sample Opinion Strategy: Use Your Imagination
We're constantly surprised by the wild and wonderful ways that OpinionX stack ranking surveys have been used by students, product teams, political parties, entrepreneurs, non-profits, YouTubers, and many others to date.
Keep coming up with wonderful ways to use stack ranking to compare things, make better decisions and empower others to be part of the process. It gives us immeasurable joy to hear about stories like these :)
Go and put these strategies to use!
It takes less than 2 minutes to create an OpinionX account and your first stack ranking survey. If you haven't given it a go already, create a stack ranking survey for free right now.
Don't be afraid to say hello if you have any questions along the way! 👋