Incrementality: what, why, and how to?

With budget constraints tightening and increasing economic uncertainty, marketing measurement is the guiding light for all those within the industry, empowering the marketing team to make data-driven decisions. 

The vast array of topics under this umbrella term (MMM, attribution, incrementality, etc.) allow us to confidently demonstrate the value of ad spend, better allocate resources to the campaigns that are driving real business results and in turn, generate more bang for the buck. So it’s no wonder marketing measurement is hot right now.

But ultimately, regardless of whether the economy is soaring or stagnating, savvy marketers are the ones that use data to stay ahead of the curve. And I’ve already written enough about measurement with attribution and using metrics like return on ad spend, so let’s dive into the next topic: incrementality.

What is incrementality in marketing?

There are two ways to throw your advertising money away: advertising to people that will never buy from you and to people that would buy it regardless of your ads.

How can you understand which conversions would have happened regardless of the ads? Enter, incrementality.

Incrementality is a marketing concept that refers to the true impact of a marketing campaign on a specific outcome, such as conversions or website traffic. In essence, it measures the difference between what would have occurred naturally and what occurred as a result of the campaign.

Incrementality measurement involves comparing the outcomes of a control group (not exposed to the campaign) to the outcomes of a treatment group (exposed to the campaign), and then calculating the lift in outcomes that can be attributed to the campaign. This lift in outcomes is what is referred to as the incremental impact of the campaign.

So now that you have an overview of what it is, let me preach why all marketers should be considering the incrementality of their campaigns.

Why measure incrementality?

Now let’s all go and shout from our office rooftops “it’s about contribution NOT attribution.” And if you don’t agree, hopefully the following examples and discussions on incrementality measurement will help to convince you.

When marketers chase after metrics such as ROAS, they are potentially optimizing on conversions that might have happened regardless of the advertising. And when marketers are fixated on attributing conversions to specific channels, they fail to consider the nuance in how much that channel actually contributed to the user converting.

Let’s examine two different scenarios to demonstrate the value of understanding and measuring incrementality.

1. Andy is looking to buy a new pair of sneakers. He knows that he likes Nike shoes and wants to purchase a new pair just like his old ones. He goes to Google and searches for “Nike sneakers”. He sees a Google ad from Nike at the top of the page and clicks on it before proceeding to make a purchase. Now, what if that ad hadn't been there? Andy was set on Nike anyway, so without a paid ad, he would have clicked on the organic result on the page and purchased it anyway. 
2. Becky has been browsing for a new mobile phone for a few days now. She has done a lot of research and decides she likes the latest iPhone. She just wants to wait until payday before she purchases. Because Becky has spent so long researching iPhones, she is in the retargeting list for their ads and is getting inundated with ads on Facebook. When payday comes around, she makes the purchase, and Facebook gets all the credit. But without any advertising, she still would have bought that phone.

The ROAS of the campaigns in the above examples might be high, but if we measure the incrementality of those campaigns, we would likely see a whole different picture. In both scenarios, the conversions would have occurred naturally. 

And if we were to use this knowledge that incrementality of the campaigns was low, we could pause them (or reduce budget) and channel the spend elsewhere for the same number of overall conversions. 

The concept of incrementality is not new, but it has gained increased attention in recent years as marketers seek to understand the true impact of their efforts and justify their spend.

Related reading: Why modern marketers shouldn't focus on ROAS alone

Distinguishing incrementality from attribution

Attribution and incrementality are often lumped together, and this can result in some confusion between these two distinct concepts.

Attribution models answer “which campaign(s) led to this conversion.”

Incrementality, on the other hand, answers “how many conversions would have happened without any advertising,” emphasizing the importance of measuring the impact of marketing campaigns. 

Let’s jump back to Becky and Andy.

1. Andy’s conversion would have been attributed to a Google brand campaign. However, incrementality measurements would have seen that the campaign did not contribute to that conversion.
2. Becky’s conversion would have been attributed to a Facebook retargeting campaign. Again, incrementality measuring would help us realize that the campaign was actually wasting spend in this case, because Becky would have purchased anyway.

Incrementality measurement is an incredibly powerful tool to help marketers save on those ever-valuable dollars of ad spend, whereas attribution is a more one-dimensional view on what ad interactions the user had before purchasing.

Related reading: Multi-touch attribution models explained

Deep dive into incrementality testing

The first step to measuring incrementality is deciding the level at which to measure. For example, the incrementality of campaigns or channels or platforms. 

The goal is to then determine how many conversions can be truly attributed to the campaign, and how many would have occurred naturally. There are several methods for measuring incrementality, including:

A/B testing

This involves splitting users into two groups. Group A is the control group that is not exposed to the campaign, and Group B is the experiment group that is. This is hard to achieve without proper tools, though, as you have limited control over ad delivery. A/B testing is often carried out through ad platforms to adjust ad delivery.

Geo testing

To start, find two similar markets. Look for similarities in size and performance (CPCs, CvR, conversion volume). One of your markets will be a control and the other will receive advertising. At the end of the month or longer, compare performance in the two markets.

Time series analysis

This involves analyzing your data over the past few months to identify trends and patterns. By comparing the trend in conversions before and after the campaign, the incremental impact of the campaign can be estimated.

Machine learning models

You might be familiar with MMM (marketing mix modeling). This is a type of machine learning model, which can also be used to measure incrementality. These models can analyze large amounts of data to determine the impact of a marketing campaign beyond what would have happened in the absence of that campaign.

No matter which method is used, it's important to ensure that the sample size is large enough to be statistically significant.

Measuring incrementality is not always easy, but it's essential for understanding the true impact of a campaign and making data-driven decisions.

Related reading: Adtriba's MMM experts reveal their secrets

Mastering the art of interpreting incrementality results

Understanding how to calculate incrementality is crucial. The formula for calculating incrementality depends on the specific method used to measure it. However, the general approach for calculating incrementality involves comparing the number of conversions in a control group (not exposed to the campaign) to the number of conversions in a treatment group (exposed to the campaign). 

Incremental conversions are then the additional conversions that the treatment group got compared to the control. 

Incremental lift formula

The formula below is a good starting point:

Incremental Conversions = Conversions in Treatment Group - Conversions in Control Group

Now, if you want to take the ad spend into account, then you can play around with CPA using incremental conversions instead of the normal total. Note that you can apply the same logic for revenue too.

Time for an example

You run a geo test in France and the UK, because they are similar sized markets and have similar CPAs and CvRs.

France receives advertising (social retargeting campaigns) and the UK receives none. The test runs for one month to ensure you have enough data.

At the end of the month, you look into the numbers. France had a total of 982 conversions, while the UK had 113. 

That means advertising led to a total of 869 (or 982 - 113) incremental conversions. 

We can assume that around 113 of the conversions that happened in France would have happened anyway.

You spent a total of €10,000 in France during that time. And with this figure, we can calculate the incremental CPA. 

10,000 / 869 = €11.51 per conversion. 

Is this still profitable for your business? How does this CPA compare to other campaigns? Can you run similar tests with different campaigns?

It's your turn to jump into incrementality testing

As a marketer, you know how important it is to measure the impact of your campaigns. After all, you (and your bosses looking in on the results) want to make sure you're getting the most out of your marketing budget! That's where incrementality comes in. It's a way of testing your campaigns against a control group to get a better understanding of how much of a difference they're really making.

By incorporating rigorous incrementality measurement, you can get more accurate insights into the true lift your campaigns are providing. And the great thing is, you can use these insights to optimize your marketing strategy and specific campaigns for even better results. When you know which tactics are working best, you can double down on them and make sure you're getting the most bang for your buck.

So now is the time to really live the motto “it’s about contribution not attribution” and go out into the marketing world with this new data tool under your belt.

Related reading: 2023's top marketing KPIs explained