How to do Mixed Methods Research without being a Quant Expert
“You don’t need a PhD in statistics to become a mixed methods researcher. If you’re a qualitative UX researcher, a handful of core quantitative skills covers most of what you’ll be asked to do: surveys and analytics for the basics, quantitative usability testing and A/B testing to level up, and statistics if you want to go deep.”
Mixed methods research blends qualitative depth with quantitative scale, and it comes in three shapes: exploratory (qual then quant), explanatory (quant then qual), and dynamic (both at once). Product-Led Growth has made this blend the norm, because when the end user is the buyer, understanding their experience with real data is what wins.
The way companies think about user experience has changed. The end user now makes the initial purchase decision to solve their own pain, not an executive thinking of bottom-line impact.
This is called Product-Led Growth, and it's shaking up tech, particularly UX and product teams. If the buyer is now the end user, then user experience is your new salesperson. There's less tolerance for bad UX than ever.
Understanding end-user pains is a discovery-driven research process that leans heavily on qualitative methods. But in nearly all cases, tech companies want quantitatively-justified decisions. That conflict between qualitative research and quantitative appetite has driven renewed interest in opening up a blend of the two, aka mixed methods research.
Companies like Meta, Amazon and Microsoft all recruit mixed methods researchers. If they ask a new research hire to jump from user interviews into quantitative analysis, they don't want to hear a meek "But that's not part of my research specialism." That's not how these companies work. They want results.
If you're a user researcher who leans heavily on a qualitative skillset, you're probably thinking "I did all this work to become a qual specialist. How am I going to learn this complicated quant stuff?" Quant can feel pretty alien and intimidating to a people-driven qual researcher. But you don't need a PhD in statistics to become a mixed methods researcher. There are only a handful of core skills you really need.
| Skill | Level | What it's for |
|---|---|---|
| Surveys | Master the basics | Validate and measure qualitative insights at scale |
| Analytics | Master the basics | Track user behaviour to spot audience patterns and issues |
| Quantitative usability testing | Level up | Measure task performance and satisfaction alongside observation |
| A/B testing | Level up | Let user behaviour decide between versions, removing designer bias |
| Statistics | Go for gold | Uncover hidden correlations and enable segmentation and personalisation |
🥉 Master the basics
Pareto's 80/20 rule says that 20% of your effort accounts for 80% of your results. This applies to learning new skills too. Master the basics and you'll be covered for the majority of day-to-day mixed methods requirements. This won't apply at every company, but it's a great place to start.
1. Surveys 🥉
Conducting and analysing quantitative surveys is the core of mixed methods. Online surveys help you validate and measure the insights you gathered from qualitative research like user interviews.
Surveys come with a big disclaimer, though: it's easy to create a survey but hard to write a quality questionnaire or gather accurate, actionable insights. We've all sat through a terrible survey, so I don't need to labour the point. Be careful out there.
Analysing surveys can seem straightforward with the automated dashboards on most survey platforms, but be warned: bias and poor representation are the carbon monoxide of surveys. They'll get you if you're not aware of them.
“Quantitative UX research, just like qualitative, delivers insights about people. All user research is human-centric; don’t lose sight of this as you start diving deeper into the numbers.”
The first step to better surveys is writing good questions: ones that steer clear of your own biases and don't lead respondents toward predisposed answers. Suggested reading: How to Create Effective User Surveys.
To be confident that your results represent the user segment you say they do, you'll need to learn how to calculate sample sizes. This is often the biggest area of friction between user research and product teams. Understand how sample-size robustness, margin of error and participant screening work. Here's a good starting point: Determining Sample Size.
Once you're ready to analyse results, HubSpot's How to Analyze Survey Results Like a Data Pro can guide you through the process.
One survey format worth knowing is Customer Problem Stack Ranking, a research method coined by Shreyas Doshi for stack ranking people's priorities to inform better decisions with real data.
2. Analytics 🥉
“Without big data analytics, companies are blind and deaf, wandering out onto the web like deer on a freeway.”
Analytics platforms like Mixpanel and Google Analytics make it easy to track user behaviour without writing any code. Installing tracking codes on your website is straightforward, and you can start collecting data and generating insights from the get-go.
Analytics is a rabbit hole you can easily get lost in for hours. Set clear goals to avoid getting distracted by shiny but unactionable insights. As an analytics newbie, focus on audience and issue identification.
Analytics tools help you segment users by criteria like demographics, device and behaviour. That lets you build more detailed personas and better understand segment commonalities. Google Analytics Academy offers free mini-courses with everything you need to get started.
Digging into metrics like bounce rate, drop-off points and rage clicking helps you identify undiagnosed issues in your user experience. Mixpanel is widely used for this: it's great for tracking millions of events to understand how individual users interact with your product. Start with their free course, Introduction to Mixpanel.
🥈 Level up
Surveys and analytics are the most common quant methods used in mixed methods. If you'd rather not move beyond these two, the examples further down show these basics in action. Otherwise, read on for intermediate methods that position you as an equally capable qual and quant researcher.
3. Quantitative usability testing 🥈
Quantitative and qualitative usability testing cover similar ground: both ask users to perform everyday tasks on your product. Quant usability testing focuses on measurable things like user performance (e.g. time spent on a task, or the percentage who completed it) or user perception (e.g. satisfaction ratings). You can gather these metrics alongside observational insights, blending qual and quant.
Unmoderated usability testing is another quantitative method. Instead of facilitating a session yourself, unmoderated tests observe larger sample sizes. You can set them up through platforms like UserTesting.com, or by analysing existing behaviour data through a platform like Mixpanel.
4. A/B testing 🥈
“The goal of a test is to get a learning, not a lift. With enough learnings, you can get the real lift.”
Start with a goal in mind (e.g. "I want to increase conversion rate"), create a control and a variable version of what you're testing, and watch the data to see which performs best. You can make A/B testing more advanced by adding more variable versions and testing across different user segments.
A/B testing breaks down into a five-step process: (1) identify a goal, (2) form a hypothesis, (3) design and run the test, (4) analyse the results, (5) implement the results.
A/B testing enables incremental improvements that add up to a big difference over time. The best teams methodically use A/B tests to optimise landing pages, buttons, copy and more. It removes designer bias by handing product decisions to users, via their behaviour.
Researchers rarely run A/B tests on their own, so understanding how they work will help you collaborate with the product or development teams who typically own these experiments. UX Booth has a great 5-step guide to getting started, and Smashing Magazine's Ultimate Guide to A/B Testing goes deeper.
🥇 Go for gold
Here we cross to the other side of Pareto's 80/20 rule: the methods that take 80% of your time to master but only account for 20% of your quant requirements as a mixed methods researcher. These aren't for everyone. If you'd rather learn how to combine the topics already covered, jump to the examples. If you're up for a challenge, I salute you. ⏩
5. Statistics 🥇
Most qual-first researchers have fled in fear by now. But if you decide to become an expert Full-Stack Researcher, a foundation in statistics matters.
Statistics take you deeper than generic tools can. They can uncover hidden correlations and relationships between behaviour data points and inform segmentation decisions. At best, statistics let you move beyond segmentation altogether toward true user experience personalisation.
To dip your toe into the statistics useful for UX, consider Quantifying The User Experience: Practical Statistics For User Research by Jeff Sauro. For a quicker overview, Andy Park's Using statistics in UX design is a good read. And for hands-on, skill-based learning, give SPSS a go with Discovering Statistics Using IBM SPSS Statistics.
🏆 Mixed methods research designs
The best way to test your new quant knowledge is to try a mixed methods project. There are three main types of mixed methods: exploratory, explanatory and dynamic.
Exploratory mixed methods (qual → quant) 🔭
Exploratory mixed methods puts the qualitative step first to establish foundational knowledge about your topic. It's particularly helpful when you have a lot of unknown unknowns.
A common approach is to run user interviews to investigate a topic in depth, gathering opinions, perspectives, motivations and unmet needs. The hypothesis from this qualitative step informs your quantitative survey questions. The survey results then let you measure and validate your qual insights with a statistically significant sample size.
Check out a real-life exploratory mixed methods example here.
Explanatory mixed methods (quant → qual) 🔍
Explanatory mixed methods puts the quantitative step first. It's especially useful when you've already run a quant project and need to understand unexplained or surprising insights.
A common approach is to analyse user analytics or survey data. During this step you'll often see what people are doing but struggle to understand why. Bring those "why" questions into a qualitative deep-dive: in an interview, participants can share the reasons behind their actions, giving you a fuller understanding of the topic.
Check out a real-life explanatory mixed methods example here.
Dynamic mixed methods (qual + quant) 🔮
Dynamic research beats the time-consuming side of mixed methods by blending qualitative and quantitative data at the same time. A whole mixed methods project can run in a single process with a dynamic research tool, unlike the multiple steps exploratory and explanatory research need.
For example, OpinionX enables dynamic mixed methods for discovery research by letting participants vote on each other's opinions, adding a quantitative dimension to otherwise unstructured qualitative input. That saves the researcher from manually taking insights from the first step and translating them into a different research method, which can eat a lot of time.
The result is a clean blend of qualitative user opinions that are easy to prioritise with quantitative sorting based on consensus and importance.
Check out a real-life dynamic mixed methods example here.
👉 Learning as you go
The UX research industry has been heading toward mixed methods for years, and Product-Led Growth has only accelerated the trend. Adapting your skillset to include mixed methods can feel intimidating, but you don't need a formal background in quantitative research to get started.
Through the resources linked above (bookmark this page to revisit them), along with a bias towards hands-on experimentation and trial and error, you'll be on your way to becoming a Full-Stack Researcher in no time.
👊 Written by Éamon Cullen and Daniel Kyne from OpinionX.
Frequently asked questions
What is mixed methods research? Mixed methods research blends qualitative methods (like user interviews) with quantitative methods (like surveys and analytics) in a single study, so you get both the depth of qual and the scale of quant when making a decision.
Do you need to be a statistician to do mixed methods research? No. You don't need a PhD in statistics. A handful of core skills, surveys, analytics and A/B testing, covers the majority of day-to-day mixed methods work. Statistics only becomes important if you want to go deep into correlations and personalisation.
What are the three types of mixed methods research? Exploratory (qualitative first, then quantitative to validate), explanatory (quantitative first, then qualitative to explain the "why"), and dynamic (both at once, blended in a single process).
What quantitative skills does a UX researcher need? Start with surveys and analytics, then add quantitative usability testing and A/B testing. Statistics is the optional advanced skill for those who want to become a full-stack researcher.
🙌 Bonus tip: immerse yourself in online communities
UX research is one of the fastest-changing functions in tech, so staying on top of best practice helps you explore the topic deeper and get the most from your findings. One of the best ways to stay current is by following the experts and joining active user research communities. Get an email every two weeks on the future of user research by subscribing to our newsletter.