Points Allocation Survey Method [Explanation & Real Examples]
Contents:
→ What is a Points Allocation survey question?
→ What’s an example of Points Allocation in a real research project?
→ When is Points Allocation the right question to use in a survey?
→ What research questions could I answer with Points Allocation?
→ Which survey platforms offer a Points Allocation question type?
What is a ‘Points Allocation’ survey question?
Points Allocation (also known as Constant Sum) is a survey method where respondents allocate a budget of points amongst a set of options according to their personal preferences. Here’s an example of a Points Allocation question:
Points allocation is a great way to rank a set of options because it allows respondents to show the magnitude of their preferences, not just their ranked order. For example, we don’t just learn that Simon prefers apples to bananas, we see that he would give 9 of his 10 points to apples.
What’s an example of Points Allocation having real impact in a user research project?
I bet you didn’t know that Points Allocation played a huge role in the early success of Google Docs!
Back when Google Docs was still in its beta phase, the team was working with pilot users to improve the product experience before public launch. As a beta product, it was still pretty rough around the edges, so the team was collecting user feedback about which aspects of the product needed improvement the most. They collected suggestions from users during informal interviews and testing sessions but the tallied results showed a large range of features users felt were needed, leaving the Google Docs team lost as to where they should focus their efforts — until they ran one simple experiment.
A Product Manager on the team asked beta users, “If you had $100 to spend on improving Google Docs, how would you spend it amongst this list of problems?”. The result was mind-blowingly clear — beta testers allocated 89% of all points on document formatting.
^ Google Docs, October 2006 (lacking a lot of formatting…)
They took this insight into user interviews and asked testers to explain their problems with document formatting in more detail. They realized that minor text changes could destroy an entire document’s format, requiring hours of work from the user to fix. This was a big enough problem that it often made beta testers rage quit and give up using Google Docs entirely.
The Google Docs team had discovered an important fact about user research — asking users to suggest problems is unreliable because it relies on problem recall at that exact moment, which people are notoriously bad at. Instead of counting a tally of feature/improvement requests, Points Allocation allowed the team to measure the magnitude of problem impact to their beta testers, surfacing the highest leverage problem to solve.
When is Points Allocation the right question to use in a survey?
A ranking format like Pairwise Comparison (ranking by voting on a series of head-to-head pairs of options) surfaces people’s implicit preferences by showing what matters to them without asking them to directly say so. Points Allocation, on the other hand, shows someone’s explicit preferences by asking them to indicate what matters to them and how much.
Implicit formats like Pairwise Comparison are a great way to understand people’s decision-making when their subconscious mind is in the driver’s seat, like ranking a list of potential names for your new idea, where it’s hard to rationally compare multiple options at once. Explicit formats like Points Allocation are superior for simulating rational decisions, like ranking which features in a paid subscription plan drive the most value for you as a customer.
What research questions could I answer with Points Allocation?
Here are three examples of survey questions that are suited to the Points Allocation format:
1. Customer Preferences
Like in the Google Docs example, Points Allocation is a great way to understand the magnitude of people’s preferences. If you’ve got a range of different options that you’re trying to decide between, like which features most entice your freemium users to upgrade to a paid subscription, then Points Allocation is often a more insightful ranking method than alternative comparison formats.
2. Value Association
Imagine you run a business that offers multiple services rolled into one — for example, you’re a graduation photographer who charges $50 to edit the photos, mail physical prints, and share digital copies for future use. You’re thinking about splitting these into separate products to increase your prices, but you don’t know which ones customers value most today. Giving participants $50 in imaginary points to divide based on how much they’d expect to pay for each separately will surface target customer’s perceived values.
3. Time Breakdowns
Casey Neistat had a famous YouTube video in 2015 called Fat and Lazy where he broke his average day up into 24 one-hour slots to show viewers what he spends his time on.
You can do this same exercise with your research participants using a Points Allocation question. The average American office worker spends 37 hours per week working, so give each participant 37 points and ask them to show what they spend their time on.
Which survey platforms offer a Points Allocation question type?
Surprisingly, almost none of the top survey tools offer Points Allocation questions — that includes Google Forms, Microsoft Forms, Typeform, SurveyMonkey, Slido, SurveyKing, or Jotform. None of them! Here are some free survey tools you can use for your Points Allocation questions…
Alternative 1: OpinionX
All of the example screenshots from this post were taken from OpinionX — a research tool built specifically for creating ranking surveys. OpinionX is used by thousands of companies (including Google, Amazon and Microsoft). All of OpinionX’s various ranking formats, including “Points Rank”, are available on its free version.
^ “Points Rank” is the name for Points Allocation questions on OpinionX
Free Version: The free version of OpinionX gives you unlimited surveys with unlimited questions and unlimited ranking options. All question types are available on the free version and you can have up to 50 participants per survey (no monthly/annual limits, just per survey).
Because OpinionX was built specifically for ranking, it comes with a bunch of purpose-built features for analyzing ranked data — for example, the segmentation feature lets you recalculate your results to see how different subgroups of your respondents voted compared to your overall results.
Premium Tiers: OpinionX’s “Pro” tier (100 participants per survey) costs $30/month and offers extra customizations like removing OpinionX branding or setting a minimum/maximum number of votes per participant. Advanced analysis like survey branching and results segmentation comes on the “Scale” tier (1,000 participants per survey) which cost $60/month. Check out the full OpinionX pricing breakdown for more info.
Alternative 2: Mentimeter
Mentimeter is an audience engagement tool built for classrooms that has a question format it calls “100 Points”, which gives participants 100 points to allocate — but only in bundles of 10. It’s not possible to change it from 100 points or the 10-vote bundles. Also, its free version has a limit of two questions per “presentation” and premium plans are charged per-year only.
Alternative 3: Polly
Polly is an app for running polls in Slack, Microsoft Teams and Zoom. It has an option for Points Allocation surveys, although they are limited to 10 options and a maximum budget of 100 points per participant. The free version allows for 25 monthly responses across all polls/surveys.
Create a Points Allocation Survey
You can have your first Points Allocation survey up and running in less than 5 minutes on OpinionX. There’s no time limit to the free version and you don’t need to add your credit card details — the free tier genuinely is just free — so there’s no reason not to give it a try! Create a Points Allocation survey on OpinionX today.