In the first installment of our new partner video series, WEVO’s Jenni Bruckman was joined by digital optimization rockstars Beth Damiano, Digital Marketing Consultant at Accenture Interactive, and Cole Ranzau, Supervising Digital Marketing Manager at Strategic Education, Inc.
From the perspectives of an agency partner (Beth) and client (Cole) who’ve worked together to optimize digital campaigns through user testing, they discussed with Jenni the powerful insights that can be uncovered “when qual and quant collide.”
Watch the full episode and read on for key insights from their conversation.
When Conducting A/B Tests, Learn from the Losers
“When we put a digital experience in front of customers and the data shows very clearly that we did not improve the experience, it’s beneficial to be able to understand what they don’t like, just as much as it’s important to understand what they do like. Both of those elements help impact the business on all different fronts, from knowledge of our user base through that end goal of leads or revenue.” — Beth
A/B testing is a tried-and-true method of establishing which of two concepts a group of users prefers. It’s a great way to gather quick, quantifiable feedback before launching a new digital experience. However, focusing only on the statistical winner of an A/B test can mean missing out on valuable insights.
The key to extracting insights from a poorly-performing digital experience is augmenting your A/B approach by adding qualitative data to pinpoint the reasons that it didn’t resonate.
“Not every test is going to move the needle from a conversion perspective, but it’s important to leverage those learnings to propel you into the next great idea or the next big strategy. It’s easy to see that an experience didn’t work, but really digging in [with qualitative data] and understanding why it didn’t work is the key to understanding how to improve it.” — Cole
Always Be Testing, Always Be Learning
“I love this mantra because what works for us today is not necessarily going to work for us a year from now, or even a month from now. The world is always changing. You could test something today that doesn’t move the bottom line, but it’s definitely something that could be worth testing again, six months or a year or two years from now, depending on how things shift.” — Beth
Testing is not one-and-done. Successful digital optimization is built on a process of continuous iteration and experimentation. Customer expectations aren’t stagnant; in fact, they’re evolving faster than ever, and the best way to keep a pulse on your market is to create touchpoints through frequent user research. This doesn’t mean, though, that testing has to be high-stakes. Instead of using your live traffic as guinea pigs for every new idea, mitigate risk with pre-live testing that provides the statistical significance to validate your concept so you can launch with confidence.
“It’s not always a comfortable place for everyone in the organization to take what’s already working and improve on it. Sometimes in doing that, there will be some missteps. But learning from those missteps will get you to that next step. The safety net of testing is that you’re not just putting a brand new experience out there. You’re doing it in a very calculated way.” — Cole
Qual + Quant = The Full Story
“Qualitative is what optimization is all about. It’s one thing to see improvement or decline in the quantitative data, but the qualitative can help you answer why we are seeing what we’re seeing. It helps you create a more informed user-centric experience.” — Cole
Given a choice between qual and quant, it’s easy to default to the latter — it’s often easier to analyze and we have confidence in the numbers. But quantitative data alone is only part of the story. Qualitative data provides context and meaning for a more complete understanding of how the digital experience meets, or falls short of, the customer’s expectations.
“I always think of this as a three-pronged attack of the web data or quantitative research to help you show the what, the qualitative research to help you show the why, and the A/B testing to show how to implement it. That what, why, and how are so important to moving the needle for you and your business.” — Cole
“Qualt” for the Win
In their work together, Beth and Cole have leveraged the combination of qualitative and quantitative data to uncover insights that led them to success with engaging, high-performing digital experiences for Cole and SEI.
“Cole and I ran an A/B test together that stemmed from an initial bout of qualitative research. Then we tested that through A/B testing to a larger audience and it didn’t work so well. We were able to pair the initial findings with our quantitative data. We pivoted that approach and launched a variant of that first test and it turned into a large winner for us. That is the prime example of being able to marry the two together to find what is that optimal experience.” — Beth
Learn more about the power of qualt and how WEVO can help you harness it to optimize your digital experiences.
