7 Group Voting Methods For Ranking Any List of Ideas
โIdea ranking turns a messy list of suggestions into a clear, agreed order your team can act on. The best methods force people to compare options instead of rating each one in isolation, because a 1-5 rating lets everyone score everything highly and separates nothing.โ
There are seven idea ranking methods to choose from: pairwise ranking, ranked choice voting, points allocation, MaxDiff analysis, dot voting, image voting, and consensus ranking. Pairwise ranking (a series of A-vs-B choices) is the most flexible and the best pick for long lists of 10 or more options, where drag-and-drop rank ordering gets error-prone. Which one fits depends on three things: the device people vote on, how long your list is, and whether you need to compare results across multiple teams or segments.
Topics covered:
I've helped teams run idea ranking projects with millions of votes cast, so I've learned a thing or two about the mistakes to avoid. Ranking surveys sound simple, but a badly run one can undermine your whole project or spark disputes and grudges among teammates that are hard to resolve.
7 idea ranking methods for your team
| Method | How it works | Best for |
|---|---|---|
| Pairwise Ranking | Head-to-head A-vs-B votes, scored 0-100 | Almost anything, especially long lists of 10+ |
| Ranked Choice Voting | Drag-and-drop the list into order | Short lists of 5-10 options |
| Points Allocation | Spend a budget of points across options | Showing the magnitude of preference |
| MaxDiff Analysis | Pick best and worst from small sets | Ranking many items by importance |
| Dot Voting | Distribute dots across options | Quick in-person or whiteboard voting |
| Image Voting | Pairwise votes on images | Ranking visual ideas or mockups |
| Consensus Ranking | Agree or disagree with each statement | Finding what everyone agrees on |
1. Pairwise ranking
My favourite format for idea ranking is pairwise ranking, which breaks a list of options into a series of head-to-head pair votes. It's ideal for almost any scenario, whether you're group voting or just ranking your own preferences, and it works for long lists of 10+ options as well as short sets.
Unlike other methods, pairwise ranking brings out people's implicit preferences. When you're voting on ideas, working out what you actually prefer takes real mental effort. Breaking the list into "A vs B" decisions gives participants an easy way to show what they prefer.
You can create free pairwise ranking surveys on a tool like OpinionX, which picks the pairs for you and scores results on a 0-100 scale. You can even set how many pairs each participant votes on.
2. Ranked choice voting
Rank order questions are the most common ranking exercise people do online. Participants drag and drop a list of options into their preferred order, then submit. But most survey providers recommend rank order only for lists of 5-10 options. For longer lists, pairwise ranking is a better fit.
3. Points allocation
Points allocation questions (also known as Constant Sum) ask participants to spend a budget of points among a set of options by preference. It shows the magnitude of a preference, not just the ranked order. We don't just learn that Simon prefers apples to bananas, we see he'd give 9 of his 10 points to apples.
Points allocation is basically the opposite of pairwise ranking. Its results come from participants' explicit preferences (their awareness of what they like), so it suits decisions with an explicit, rational side, like purchase motivators, more than ones that draw on implicit criteria, like idea ranking.
4. MaxDiff analysis (best/worst scaling)
MaxDiff ranks preferences by asking people to pick the best and worst option from a set of 3-6 statements. Each vote shows a new set drawn from your full list. You can relabel "best" and "worst" to fit the study: Favourite / Least Favourite, Top / Bottom, Most Preferred / Least Preferred.
5. Dot voting
Dot voting is the traditional version of points allocation. Each participant gets a number of "dots" to spread among a set of options, either on a virtual whiteboard or with in-person posters and stickies, which is why it's common in focus groups. It's quick and hands-on, but you have to tally it by hand and can't ask any follow-up questions.
6. Image voting
Mockups, sketches, and drawings get an idea across faster than text or speech alone. Let team members submit simple images for their ideas, then use pairwise ranking to turn those images into ranked results.
7. Consensus ranking
Sometimes what everyone agrees on matters more than the single highest-voted idea. Agreement ranking shows participants a statement and asks them to agree or disagree. It's a well-studied method used in public consultation projects like the vTaiwan participatory democracy project.
What scenarios should change the method you pick?
1. Participant device
As of 2022, over 56% of all surveys were answered on mobile, but our small phone screens are pretty terrible for most ranking questions (especially ranking images). Pairwise ranking suits mobile participants well because it only shows two options at a time.
2. List length
The most common ranking format, drag-and-drop rank ordering, is really bad at long lists. SurveyMonkey says don't use it past 5 options, Qualtrics says 6-10 max, and Slido vaguely says "less is more". A 2016 study found rank order questions get more error-prone and time-consuming the more options you add. Past 10 options, switch from rank order to pairwise ranking.
3. Multiple teams or groups
If you're running one survey across multiple teams, groups, cohorts, or segments, plan for it. First, collect data that identifies which group each participant belongs to (easily done with multiple-choice questions). Then you can segment and compare the ranked results for each group.
In more extreme cases you might need a different ranking list per team. Use survey branching to send participants to different ranking questions based on their multiple-choice answers.
^ quick example that shows why itโs very important to consider segmentation when analyzing any ranked results
How do you pick the right idea ranking tool?
Every ranking method in this post is free on OpinionX, a research tool for ranking people's priorities. If you're weighing the methods against each other, this guide to choosing a survey ranking method walks through the decision.
Create and launch your own ranking exercise on OpinionX in under 5 minutes. No credit card, no time limit, and you can create as many surveys, questions, or ranking options as you like on the free tier.
Best-practice principles for idea ranking exercises
1. Comparative. The best kind of ranking is comparative. Instead of asking people to rate each option 1-5 stars, it forces them to compare options and decide which matter most. All the methods here except method 7 are comparative.
Avoid: rating methods like 1-5 Stars or Likert Scales.
Adopt: comparative methods like pairwise ranking.
2. Crowdsourced. The best ranking exercises start well before any votes are cast. For real participation, let your team share ideas, add to the list of options, and shape the discussion.
Avoid: shallow or performative participation.
Adopt: crowdsourced ranking lists.
3. Countable. With any research or engagement project, think about what output you'll need for the decisions that follow. Almost always, the best output from a ranking exercise is a numeric, easy-to-read dataset.
Avoid: vague, unactionable, or purely qualitative outcomes.
Adopt: quantitative or mixed methods ranking.
4. Collective. You're probably giving your team a vote because you want buy-in and ownership of the outcome. The easiest way to ruin that is to give some participants more influence than others. You can still isolate or compare segments while keeping it fair (e.g. segmentation filters during analysis).
Avoid: exclusive or unequal participation.
Adopt: one person, one vote.
Frequently asked questions
What's the best way to rank a list of ideas? Pairwise ranking is the most flexible idea ranking method and works for almost any case: it breaks the list into simple A-vs-B choices and brings out people's implicit preferences. For a short list of 5-10 options, drag-and-drop rank ordering is quicker; beyond 10 options, pairwise ranking is more reliable.
What's the difference between pairwise ranking and points allocation? Pairwise ranking captures implicit preferences by making people choose between two options at a time. Points allocation captures explicit preferences and their magnitude, showing not just that apples beat bananas but by how much.
How do you rank a long list of ideas? Don't use drag-and-drop rank ordering past about 10 options; it gets error-prone and slow. Use pairwise ranking instead, which only shows two options at a time and works well on mobile.
Can you rank ideas across multiple teams? Yes. Collect a multiple-choice question identifying which group each participant belongs to, then segment and compare the ranked results by group. For different lists per team, use survey branching.