7 Types of Ranking Exercises For Focus Groups (Online & Offline)
“A focus group ranking exercise turns open discussion into countable, comparative data you can act on. The trick is to make participants choose between options instead of rating each one in isolation, because a rating scale lets everyone score everything 4 out of 5 and tells you nothing.”
There are four online methods (pair ranking, order ranking, image ranking, and points-based ranking) and three in-person ones (dot voting, sticky-note stack ranks, and yes/no secret ballots). Online methods scale, calculate results instantly, and let you compare how each focus group differed; paper methods are quick to set up but slow to count and harder to keep free of peer influence. For lists longer than 10 options, pair ranking (pairwise comparison) gives the highest-quality result.
Topics covered:
What should you consider when designing focus group ranking exercises?
These principles apply regardless of whether your focus group is in-person, online, or a hybrid of both.
Principle 1: Comparative
In real life, decisions are comparative; you've always got a bunch of options, trade-offs, problems, ideas, or opportunities to weigh. Ranking exercises should mirror that and make participants choose between the options, not rate each one on its own. The thing to avoid is the Likert scale (rating each option 1-5 from "very satisfied" to "very dissatisfied"). If someone can rate every option 4/5 and wreck your data, the exercise was badly designed.
Principle 2: Repeatable
Reliable research is repeatable research. That matters in focus groups, where you'll usually have to combine the data from each group into one overall result for your team. The best way to get repeatability is a structured ranking process. If you can't repeat it identically each time (e.g. with an online tool), at least hire an experienced facilitator to stop participants swaying each other's votes.
Principle 3: Countable
Ranking exercises suit focus groups because they turn qualitative insight into quantitative data you can actually decide on. If you let participants set the ranking list themselves, you risk ending up with qualitative statements you can't count. That said, later in this post I explain one way to do crowdsourced ranking with focus groups.
Here are the seven ranking exercises covered in this guide:
| Ranking exercise | Type | How it works | Best for |
|---|---|---|---|
| Pair Ranking | Online | Head-to-head pair votes, scored 0-100 | Long lists (10+), mobile participants |
| Order Ranking | Online | Drag-and-drop the full list (Dowdall Method) | Short lists of 10 options or fewer |
| Image Ranking | Online | Pair votes on images | Comparing visuals, especially on phones |
| Points-Based Ranking | Online | Spend a pool of points across options | Measuring the size of the gap between options |
| Dot Voting | In-person | Stick dots or stickers on options | Quickly surfacing favourites in a big group |
| Sticky-Note Stack Ranks | In-person | Each person ranks, then compare for overlap | Aligning group priorities |
| Yes/No Secret Ballots | In-person | Anonymous written Y/N slips | Reducing peer influence on the vote |
What are the best online focus group ranking methods?
There are four online ranking methods we can use that meet all three of our design principles.
1. Pair Ranking
Pair Ranking, also known as pairwise comparison, uses a series of head-to-head votes to score and rank a list of options. Instead of dragging a long list into order, participants get easier mini-tasks that add up to the same ranked data at the end.
You can create Pair Ranking exercises free on OpinionX. Each option scores 0-100, the percentage of pair votes it won. You can set how many pairs each participant votes on, from a quick 5-vote sample to every possible pair combination.
2. Order Ranking
This is the traditional "drag-and-drop" list you'll find on many survey tools. For shorter lists of 10 options or fewer, Order Ranking is a quick way to spot people's priorities. Past 10 options, Pair Ranking gives higher-quality results.
The Order Ranking question type on OpinionX uses a scoring formula called the Dowdall Method. Each option scores 100/rank (1st gets 100, 2nd gets 100/2 = 50, 3rd gets 100/3 = 33, and so on). It's ideal because it's easy to understand and people don't all have to rank the same number of options to get a normalised score.
3. Image Ranking
The best way to do Image Ranking online is the same format as Pair Ranking: break the options into a series of head-to-head pair votes. That matters most for mobile participants (over 56% of surveys were completed by phone as of Q4 2022), where there's no room to drag-and-drop rank several large images.
4. Points-Based Ranking
Comparative methods like Pair and Order Ranking are good at relative importance, but they can hide how big the gaps between options really are. To measure that magnitude, give each participant a set number of points to 'spend' among the options. You might find the gap between your top two is bigger than you expected!
Benefits of online ranking methods
i. Scale. Online ranking works just as fast with 2 people as with 2,000, so the tooling holds up whatever your research needs.
ii. Segmentation. You can create segments for each focus group, so one survey gives you the overall result across everyone, then lets you filter the ranking to compare how each group differed.
iii. Speed. OpinionX calculates results instantly, so you can spend the spare time sharing them with the group. That opens up a discussion where people react to the overall results and you learn the "why" behind their priorities.
iv. Sophistication. Online tools have features you can't match on paper. For example, you can run a ranking survey in two steps: first collect people's written opinions, then add them to a Pair Ranking block to see whose statement best captures the room's collective view.
What in-person focus group ranking exercises can you use?
Time to break out the sticky notes.
1. Dot Voting
From 2014 to 2020, I helped organise hackathon events called Startup Weekends. Anyone pitching a product idea had to make a quick A1 poster on flipchart paper, and then we'd give every participant 3 sticky notes to vote for their favourites.
Dot voting is a quick way to find a large group's overall favourites (though counting and sorting 300+ votes in a couple of minutes takes all hands on deck). It's also a popular online method, with free tools like Lucidchart and Miro.
2. Group Sticky-Note Stack Ranks
Group stack ranks are where each person ranks their own priorities and then you analyse the results for overlap. It's a good way to move from open conversation to a deeper discussion about group priorities. At Startup Weekend hackathons, we used to get teams to run both of these on Saturday mornings to align on what to work on.
2a. Freelisted. If teams are still early in brainstorming, a freelisted stack rank helps find consensus. Each person gets 10 sticky notes, writes the 10 things they think matter most (anything they like), and sticks them in a vertical line on a blank wall or board. Freelisted stack ranks break all three of our design principles, so use them as an early shortlisting exercise, not a research method for understanding team priorities.
2b. Shortlisted. Shortlisted stack ranks work the same way, except the top 10 or so priorities are already mostly agreed. This time each person ranks the same list of 10 options from 1st to 10th. You score it by counting positions for each option (ranked 4th = 4 points), then sorting from fewest points at the top to most at the bottom. It's an inclusive way to decide in workshops, hackathons and focus groups.
3. Yes/No Secret Ballots
The easiest of these exercises is the yes/no secret ballot. Each person writes Y or N on a slip, folds it, and throws it into the middle. You count the slips (often the group helps, to speed it up), then run the next ballot.
This method hits a lot of our principles: it's repeatable, countable, and anonymous voting cuts group influence. But it's not really comparative. You can list several options and have people secretly vote for their top one (number the list 1-10 and they write the number they like most), but you lose the richer picture of their overall priorities, which matters most on divisive topics with a wide spread of opinion.
Challenges with paper-based group ranking methods
i. Repeatability. People sway each other far more in offline exercises, where everyone can see how you vote. A good facilitator can even out the personalities in the room somewhat, but it's an unavoidable part of any in-person, discussion-based setting.
ii. Scale. Paper formats take more time to count. The bigger the group or the longer the list, the more time it needs, so offline methods are hard to scale to larger groups.
iii. Time. Because analysing offline results by hand eats time, facilitators rarely get to bring the results back to the group for more discussion. That looping, self-analysing step is often where the richest insight into a group's "why" comes from.
How do you pick the right ranking exercise for your focus group?
Ranking exercises drive engagement in a focus group and give you quantitative data for later decisions. When you're paying for participant and facilitator time, you want the most value per minute, and online ranking is the most efficient way to get it. If you're weighing the methods against each other, this guide to choosing a survey ranking method walks through the decision, and points-based ranking has its own guide too.
You can create unlimited ranking surveys free on OpinionX, using every online method in this guide. Companies like Google, Shopify and Amazon trust OpinionX to measure what matters most to their customers, colleagues and communities.
Create a focus group ranking survey in under 5 minutes for free.
Frequently asked questions
What is a focus group ranking exercise? It's an activity that asks focus group participants to order or choose between options, turning open discussion into countable, comparative data. Unlike a rating scale, it forces trade-offs, so the results actually separate the options.
What's the best ranking method for an online focus group? Pair ranking (pairwise comparison) for lists of 10 or more options, because it breaks a long list into easy head-to-head choices. For shorter lists, order ranking (drag-and-drop) is quick and intuitive.
How do you run a ranking exercise in an in-person focus group? Three formats work with paper: dot voting (stickers on options), group sticky-note stack ranks (everyone ranks, then compare), and yes/no secret ballots (anonymous slips to reduce peer influence).
Why use ranking instead of a rating scale in focus groups? A rating scale lets everyone score every option 4 out of 5, so nothing separates and the data is unusable. Ranking makes people choose, which produces a clear order you can act on.
About the author:
Daniel Kyne is the Co-Founder & CEO of OpinionX, the platform for advanced market research surveys. He was previously Digital & Innovation Lead at Unilever UK, where he received a consumer research award and organised focus groups with some of the top market research agencies in the world.