Why Organizations Need to Combine Quantitative Testing and Qualitative Research
Critical business decisions are made with the help of quantitative and qualitative data. Unfortunately, that data comes from different sides of an organization. This siloed approach holds both sides back from reaching impactful results.
For many insight teams, the idea of combining quant and qual can seem too daunting. Yet many in the user research field are advocating that shifting user research left – meaning they are introducing optimization earlier. This leads to better customer feedback collection to continually build the right customer experiences.
WEVO’s Jenni Bruckman, VP of Customer Success & Strategic Partnerships, and Paula Sappington, Sr. Director of Digital Research at Hilton, recently discussed this topic. They discussed practical strategies to successfully merge quant and qual insights practices to meet organizational and consumer demand that deliver the right customer experiences. Specifically they shared insights on how organizations should:
- Combine the power of quant and qual to reduce the risks of usability, win at optimization and research, and drive an experimentation mindset across the organization.
- Empower teams to accelerate the integration of quant and qual, build a better customer experience through insight-led innovation, and accelerate impact.
- Get teams to shift left to de-risk, democratize and deliver with the capabilities of smart AI and/or human research experts.
Left-brain vs. right-brain research approaches
Presented by: Paula Sappington, Sr. Director of Digital Research at Hilton
When talking about quantitative research Paula referenced optimization experimentation programs that take an analytical approach, a left-brain approach to insights.
“It really tells us which experience performs better in the wild. When we think about AB testing, it coincides with left-brain activities. What makes people think this way? Left brain-type organizations are the data-driven, decision-making teams. These are the teams that just need to know what works best. They think in numbers, they rely on analytics platforms, which are hugely vital to operations. And ultimately, they’re going to let statistical significance rule the day. Reliability out in the wild is just the ultimate end goal.”Paula Sappington
But what is often missing for teams that focus on quantitative research methods is understanding the right tests to run. Although they might have insights into “what works,” the testing can be flawed without understanding the “why” behind the behaviors. With qualitative research approaches, teams can take a right-brain approach to identify the “why”.
“[Qualitative research] helps identify the problems to solve. Ultimately, talking to our customers can inform us what we should work on, where we can reduce friction, and what users are expecting from us, our competitors, or anyone in the space. It can inform the roadmap, and ultimately, it’s about why. Why do the customers behave and feel the way they feel? Shifting back to our brains, when we think about right-brain activities, these are organizations that really want to hear it in words and visuals. They’re convinced by conversations and they react to the strength of the feelings that are being expressed by their customers. These are typically tracked by special platforms that enable the distribution of learning, which of course, leads to democratization. Doing this work helps us understand the why.”Paula Sappington
The true magic in user research happens when organizations can combine both quantitative AND qualitative research methods – helping to inform what customer problems to solve and what to spend time on.
When to use qualitative vs quantitative research
Presented by: Jenni Bruckman, VP of Customer Success & Strategic Alliances at WEVO
Qualitative research can help inform what quantitative research to run, increasing the chances of success. Qualitative approaches allow for cheaper, more nimble risk reduction to better inform the more costly, time-intensive quantitative testing that ultimately increases your win rate, and therefore your learnings and bottom line.
When we think about traditional human-led UX research, we think about these expert practitioners – the UX researchers and UX designers working in lockstep through preparation. However, their work is very manual. They spend hours summarizing insights and succinctly bring them together, often with low sample size.
“At WEVO, we’re able to leverage AI to organize all of that data, make sense of it, and understand what the key themes are using a consistent set of lenses over and over again. But, most importantly, you don’t have to be able to dedicate hours and hours to do that. When you take rich qualitative feedback at a quantitative scale, you’re able to lean into a five-minute process to launch a test, and anywhere from 30 minutes to an hour on really understanding some clear, summarized results.”Jenni Bruckman
If teams are spending time and energy placing bets on their roadmap and iterations, how can they maximize the scale and reliability of that manual work? The solution is AI-powered insights to help expedite testing preparation and data synthesis.
Smart AI reliably automates all of that heavy lifting, so smart humans can curate and analyze the findings, and free them up to lead more complex problem-solving.
“What if you could run as many studies and as many tests as you want, asynchronously at any given time, with a larger sample size? You’re able to do this by leveraging that standardized study design. It’s a flywheel that begins to happen at a much faster pace and much more efficiently. But now it’s also able to scale to multiple studies at once. So you’re able to really break open that insights backlog of things that people are curious about. Now, you can start to understand those at a much quicker clip.”Jenni Bruckman
Mile-wide vs mile-deep research
Mile-wide research is broad, giving general insights. To spend more valuable human-led time diving a mile deeper into research, organizations need a holistic approach to all of the insights, while looking at them consistently between each experience and each test.
- Have a dashboard-level view that summarizes everything and crystallizes it succinctly so that if someone only has a moment, they can glance at it and understand the takeaways.
- Look at your reliable diagnostics consistently and benchmark experiences because you’re asking a consistent set of questions across different experiences.
- Benchmark by industry and see how experiences perform relative to the benchmarks.
- Evaluate the visualized sentiment of likes and dislikes.
“AI-powered insights are reliable to all of these teams because they’re expert-advised and expert-built UX researchers. They view it as a reliable extension of their own capability, and it’s now accessible to a much broader team of practitioners. Everyone from the UX researchers to the marketers and product owners to the strategists and designers has access to that standardized and repeatable setup. Ultimately, at the end of the day, it’s about reducing risks and making every digital experience highly effective. The benefit is that you have to bring insights earlier up the funnel. That’s the heartbeat of it all.”Jenni Bruckman
Hear the conversation with Paula and Jenni in its entirety, and learn more about the power of combining quant and qual can help your team understand the why faster.