Incrementality: what, why, and how to?

With budget constraints tightening and increasing economic uncertainty, marketing measurement is the guiding light for all those within the industry, empowering the marketing team to make data-driven decisions. 

The vast array of topics under this umbrella term (MMM, attribution, incrementality, etc.) allow us to confidently demonstrate the value of ad spend, better allocate resources to the campaigns that are driving real business results and in turn, generate more bang for the buck. So it’s no wonder marketing measurement is hot right now.

But ultimately, regardless of whether the economy is soaring or stagnating, savvy marketers are the ones that use data to stay ahead of the curve. And I’ve already written enough about measurement with attribution and using metrics like return on ad spend, so let’s dive into the next topic: incrementality.

What is incrementality in marketing?

There are two ways to throw your advertising money away: advertising to people that will never buy from you and to people that would buy it regardless of your ads.

How can you understand which conversions would have happened regardless of the ads? Enter, incrementality.

Incrementality is a marketing concept that refers to the true impact of a marketing campaign on a specific outcome, such as conversions or website traffic. In essence, it measures the difference between what would have occurred naturally and what occurred as a result of the campaign.

Incrementality measurement involves comparing the outcomes of a control group (not exposed to the campaign) to the outcomes of a treatment group (exposed to the campaign), and then calculating the lift in outcomes that can be attributed to the campaign. This lift in outcomes is what is referred to as the incremental impact of the campaign. Incremental measurement can also be done using MMM. The process involves collecting data, building a model and calculating the incremental impact of each marketing channel. 

So now that you have an overview of what it is, let me preach why all marketers should be considering the incrementality of their campaigns.

Why measure incrementality?

When marketers chase after metrics such as ROAS based on simplified or click-based attribution, they are potentially optimizing on conversions that might have happened regardless of the advertising. And when marketers are fixated on attributing conversions to specific channels, they fail to consider the nuance in how much that channel actually contributed to the user converting.

Let’s examine two different scenarios to demonstrate the value of understanding and measuring incrementality.

1. Andy is looking to buy a new pair of sneakers. He knows that he likes Nike shoes and wants to purchase a new pair just like his old ones. He goes to Google and searches for “Nike sneakers”. He sees a Google ad from Nike at the top of the page and clicks on it before proceeding to make a purchase. Now, what if that ad hadn't been there? Andy was set on Nike anyway, so without a paid ad, he would have clicked on the organic result on the page and purchased it anyway. 
2. Becky has been browsing for a new mobile phone for a few days now. She has done a lot of research and decides she likes the latest iPhone. She just wants to wait until payday before she purchases. Because Becky has spent so long researching iPhones, she is in the retargeting list for their ads and is getting inundated with ads on Facebook. When payday comes around, she makes the purchase, and Facebook gets all the credit. But without any advertising, she still would have bought that phone.

The ROAS of the campaigns, calculated on click-based MTA, in the above examples might be high, but if we measure the incrementality of those campaigns, we would likely see a whole different picture. In both scenarios, the conversions would have occurred naturally. 

And if we were to use this knowledge that incrementality of the campaigns was low, we could pause them (or reduce budget) and channel the spend elsewhere for the same number of overall conversions. 

The concept of incrementality is not new, but it has gained increased attention in recent years as marketers seek to understand the true impact of their efforts and justify their spend.

Related reading: Why modern marketers shouldn't focus on ROAS alone

Drawing attribution from incrementality

When a campaign precedes a conversion, it doesn’t always cause the conversion. That’s why attribution should be based on incrementality, and this way, it allows us to answer the big question: “which campaign contributed to a conversion?”

Attribution models answer “which campaign(s) led to this conversion.”

Incrementality, on the other hand, answers “how many conversions would have happened without any advertising,” emphasizing the importance of measuring the impact of marketing campaigns. 

Let’s jump back to Becky and Andy.

1. Andy’s conversion would have been attributed to a Google brand campaign based on click-based MTA. However, incrementality measurements would have seen that the campaign did not contribute to that conversion.
2. Becky’s conversion would have been attributed to a Facebook retargeting campaign based on click-based MTA. Again, incrementality measuring would help us realize that the campaign was actually wasting spend in this case, because Becky would have purchased anyway.

Working with attribution based on incrementality, we can get a better picture of the customer’s path to conversion, without click-based attribution skewing the results. This is an incredibly powerful tool to help marketers save on those ever-valuable dollars of ad spend.

Related reading: Multi-touch attribution models explained

How to analyze incrementality

Incrementality can be analyzed with MMM and a unified marketing measurement approach, but it can also be done with incrementality testing. So let’s look at how it’s done.

1) Incrementality testing

The first step to measuring incrementality is deciding the level at which to measure. For example, the incrementality of campaigns or channels or platforms. 

The goal is to then determine how many conversions can be truly attributed to the campaign, and how many would have occurred naturally. There are several methods for measuring incrementality, including:

A/B testing

This involves splitting users into two groups. Group A is the control group that is not exposed to the campaign, and Group B is the experiment group that is. This is hard to achieve without proper tools, though, as you have limited control over ad delivery. A/B testing is often carried out through ad platforms to adjust ad delivery.

Geo testing

To start, find two similar markets. Look for similarities in size and performance (CPCs, CvR, conversion volume). One of your markets will be a control and the other will receive advertising. At the end of the month or longer, compare performance in the two markets.

2) Other approaches

Time series analysis

Time series analysis is the cornerstone for understanding how data evolves over time. It involves identifying trends, patterns, and seasonal fluctuations. This foundational knowledge is crucial for building effective marketing mix models (MMM), which rely on historical data to measure campaign impact.

Machine learning models

Machine learning offers powerful tools for measuring incrementality. Marketing Mix Modeling (MMM) is a prime example, leveraging machine learning algorithms to analyze historical marketing data. By identifying the impact of various marketing channels on sales or other KPIs, MMM helps businesses optimize their marketing spend and predict the outcomes of future campaigns. Building on the foundation of time series analysis, MMM enables a more accurate and comprehensive understanding of campaign performance.

3) Combined approaches

For a more robust analysis, consider combining MMM and UMM: Use MMM to model the impact of marketing channels and integrate the results into a broader UMM framework.

By leveraging these powerful tools and combining them as needed, you can gain a deeper understanding of incrementality and optimize your marketing strategies.

No matter which method is used, it's important to ensure that the sample size is large enough to be statistically significant.

Measuring incrementality is not always easy, but it's essential for understanding the true impact of a campaign and making data-driven decisions.

Related reading: Marketing mix modeling experts share their secrets

Mastering the art of interpreting incrementality results

Understanding how to calculate incrementality is crucial. The formula for calculating incrementality depends on the specific method used to measure it. However, the general approach for calculating incrementality involves comparing the number of conversions in a control group (not exposed to the campaign) to the number of conversions in a treatment group (exposed to the campaign). 

Incremental conversions are then the additional conversions that the treatment group got compared to the control. 

Incremental lift formula

The formula below is a good starting point:

Incremental Conversions = Conversions in Treatment Group - Conversions in Control Group

Now, if you want to take the ad spend into account, then you can play around with CPA using incremental conversions instead of the normal total. Note that you can apply the same logic for revenue too.

Time for an example

You run a geo test in France and the UK, because they are similar sized markets and have similar CPAs and CvRs.

France receives advertising (social retargeting campaigns) and the UK receives none. The test runs for one month to ensure you have enough data.

At the end of the month, you look into the numbers. France had a total of 982 conversions, while the UK had 113. 

That means advertising led to a total of 869 (or 982 - 113) incremental conversions. 

We can assume that around 113 of the conversions that happened in France would have happened anyway.

You spent a total of €10,000 in France during that time. And with this figure, we can calculate the incremental CPA. 

10,000 / 869 = €11.51 per conversion. 

Is this still profitable for your business? How does this CPA compare to other campaigns? Can you run similar tests with different campaigns?

It's your turn to jump into incrementality testing

As a marketer, you know how important it is to measure the impact of your campaigns. After all, you (and your bosses looking in on the results) want to make sure you're getting the most out of your marketing budget! That's where incrementality comes in. It's a way of testing your campaigns against a control group to get a better understanding of how much of a difference they're really making.

By incorporating rigorous incrementality measurement, whether you’re using marketing mix modeling or combined approaches, you can get more accurate insights into the true lift your campaigns are providing. And the great thing is, you can use these insights to optimize your marketing strategy and specific campaigns for even better results. When you know which tactics are working best, you can double down on them and make sure you're getting the most bang for your buck.

So now is the time to really live the motto “it’s about contribution not attribution” and go out into the marketing world with this new data tool under your belt.

Related reading: 2023's top marketing KPIs explained