Left or right? Option one or two? A or B?
A/B testing is one of the trademark tools of digital marketers. But did you know that there are several different types of tests that fall under that umbrella?
Most marketers are familiar with a very particular style of A/B testing whereby two different ad creatives are served to the same audience. These two creatives could feature different imagery, copy, or call to action.
After a set period of time or budget spent, digital marketers review which ad performed better against a chosen performance metric. The idea is that the more successful ad will then take over as the sole creative for that tactical execution moving forward.
In fact, most modern ad platforms have built their algorithms to help facilitate this selection process for you.
However, the concept of the A/B testing can be pushed to cover a wider range of digital marketing testing methods.
In our latest Funnel Tips episode, we discuss two: before-and-after testing and geo-testing. As Alex mentions in the video above, these two frameworks provide two different levels of rigor.
Before-and-after
In the simplest terms, before-and-after testing is a form of A/B testing that compares two potential solutions across two time periods.
It’s easiest to think about this concept in the form of an example. Let’s take a simple display ad. In the original state (our “version A”), the creative features a button with a call to action to “Buy Now.” The original design uses a blue button for the CTA.
In the alternative state (“version B”), the ad creative features a bright red button.
For the month of March, the brand runs version A. Then, for the month of April, the brand runs version B. This, in theory, will allow them to see if performance was affected by the color change. They will be looking for any changes before and after the change in button color.
Of course, this can be applied to all sorts of hypotheses you may wish to test. Just like traditional A/B testing, we can compare different copy, designs, and more. This type of testing can also be implemented quickly with very little planning ahead of time.
However, despite the ease of implementation, before-and-after testing does have its drawbacks. For instance, customer buying habits may be totally different between March and April.
If you’re a candy company, you’ll likely have much higher sales volume in April as it coincides with Easter.
By looking at your before-and after data in isolation, you’re not getting the full story. When using more traditional, synchronous A/B testing, these disparities are reduced, since the two creatives are run with the same distribution at the same time.
Again, though, before-and-after can be a good tool for ad hoc testing after a creative is already in the market.
Geo-testing
As Alex mentions in the episode above, geo-testing can be more precise and insightful, but it requires skilled planning in order to effectively execute it compared to before-and-after testing.
Geo-testing allows you to either isolate location as a variable in your marketing performance, or even use two very similar locations to test the effects of variance in your marketing.
Let’s break that down.
First, we can use geo-testing to determine if the regions where an ad runs affects its performance. In this example, a marketer could take the exact same ad creative and run it in two diverse locations. In the two locations, factors like local customs, attitudes, and perceptions of the brand could all affect overall performance.
This sort of testing can pair northern or southern regions of a single country. A marketer may even want to test locations in two different countries – London versus New York, for example.
Essentially, a marketer is A/B testing the effect of two locations rather than two creatives.
Another way to look at geo-testing is by pairing two very similar regions. As Alex mentions in the video, this could pair Liverpool and Manchester in the UK. These are two distinct cities, though they are located quite close to each other and share many of the same general customs.
With two similar locations, a marketer can now A/B test aspects of the marketing approach. For instance, one location may run consistent ad distribution for three months. Meanwhile, in the other location, the ad may run on and off in pre-planned increments.
Both approaches can provide marketers with valuable insights into the mechanics of their campaign. As we mentioned, though, geo-testing requires careful pre-planning before the launch of a campaign in order to define the changes you are looking for, as well as to minimize any potential noise caused by outside factors or other campaigns that will run concurrently.
Whether you’re looking to quickly implement before-and-after testing, or spend the time to plan out some more complex geo-testing, it’s important to remember that there are many styles and offshoots of the A/B testing that marketers know and love.
If you haven’t already, be sure to check out our latest episode of Funnel Tips in the video above. If you enjoy the episode, head over to our YouTube channel, then like and subscribe to make sure you get all of Alex’s latest Funnel Tips and tricks. And while you’re there, take some a virtual coffee break with our series Let’s Talk Marketing, But First Fika!
