What is incrementality testing?

An incrementality test is a randomized controlled experiment that divides people into two groups: one that sees your marketing campaign (the treatment group) and one that does not (the control group). 

By comparing the behavior of the treatment group (who saw the campaign) to the control group (who did not), you can isolate the incremental impact of the campaign. This helps determine whether the marketing effort actually drove additional conversions, sales or other key metrics or if the same results would have happened organically.

Incrementality testing helps marketers optimize their strategies, allocate resources more effectively and understand the true ROI of their campaigns. It helps them avoid assumptions and provides a clearer picture of what is genuinely working.

How does it help marketers? 

Only 6% of advertising spend is effective, but how do you know which 6% is moving the needle? Most attribution models are flawed, which means you could be wasting budget on campaigns that look profitable but aren’t driving real growth. 

Methods like last-click attribution undervalue upper-funnel efforts, making it impossible to measure long-term impact. Privacy restrictions and data loss make tracking even harder, leaving you blind to what’s actually driving revenue. 

Comparing ad channels feels like a guessing game, and scaling budgets is a gamble. Without clear, data-backed insights, you’re making decisions on assumptions, not facts.

Incrementality testing has become the benchmark for accurately measuring advertising impact while prioritizing user privacy.

These tests can be conducted across any advertising channel (or a combination of a few). They measure impact on key metrics like revenue, profit or meaningful site and app actions that contribute to your business growth.

How Uber saved $35 million with incrementality testing

While this story is about how Uber was able to uncover nine figures in ad savings, it wouldn’t be possible without the data scientist behind the strategy.

Introducing Sundar Swaminathan

As the former Head of Brand & International Growth Marketing Data Science at Uber, Sundar played a pivotal role in revolutionizing how the company approached marketing measurement.

Sundar Swaminathan, former Head of Brand & International Growth Marketing Data Science at Uber.

He used data-driven strategies and cutting-edge techniques like incrementality testing to save millions and supercharge global growth. This change in approach couldn’t have come at a better time for Uber.

Competing with Lyft and the need for better measurement

When Sumar made the decision to add incrementality testing to his measurement mix, Uber was locked in a fierce competition with Lyft. They were spending hundreds of thousands of dollars per week on A/B testing promo and pricing strategies but were still unable to get the level of insight they needed. 

Before injecting incrementality testing into the mix, Uber relied heavily on click-based performance marketing and last-click attribution for measurement, blending paid and organic data for decision-making. Even with these limited models, there were subtle indications that Uber’s market was tapped out.

Early signs of marketing saturation

To overcome this, Uber’s US and Canada rider acquisition performance analytics team was formed, focusing exclusively on paid performance marketing channels like Meta, Snapchat and affiliates. The company noticed that customer acquisition costs were dropping, which seemed positive but raised concerns when costs later spiked unexpectedly. 

A deeper analysis revealed that major US urban centers were fully saturated — most potential riders had already signed up. Meaning further spend in these areas was unlikely to drive new demand. Despite hiring a VP of performance marketing from TripAdvisor, Uber still had not built a multi-touch attribution model and continued to rely on last-click attribution.

Identifying non-incremental spend

Uber’s team overlaid seasonality data, discovering that signups followed an identical pattern despite fluctuations in ad spend. This suggested that paid ads were not actually driving new rider growth, leading the team to question the true incremental value of performance marketing spend. A key realization emerged that Uber’s biggest paid channels might not be adding value, and budget reallocation was necessary.

During this time, a major $100 million ad fraud scandal was uncovered. Bots and fraudulent publishers had siphoned millions from Uber’s marketing budget, leading to drastic spending cuts and a legal case against the advertiser. In response, Uber scaled up its data science efforts, hiring top analytics talent to build a more sophisticated measurement framework.

The $35 million incrementality test

With strong internal support, Uber’s team ran a three-month incrementality test on Meta ads for performance marketing in the US and Canada. The test revealed that Meta ads were virtually non-incremental, confirming that continued spend would not contribute meaningfully to new customer acquisition. 

A marketing mix model validated the findings, showing that paid channels contributed only single-digit percent incremental growth.

Cutting spend and shifting strategy

Based on the data, Uber made a bold decision to turn off Meta performance marketing efforts in the US and Canada, reallocating funds to higher-impact areas like Uber Eats and driver acquisition. 

Despite concerns about risk, the move faced zero internal pushback because the data made the decision clear. 

The outcome reinforced the importance of incrementality testing as a strategic tool for budget optimization and business growth. It also shows how data can plug gaps in understanding where there are signs that things aren’t quite working, but it’s not obvious why. 

How was the $135 million reinvested?

Saving money is just one part of the marketing puzzle. The true value of these savings is determined by how they are used to drive future growth.

Because of the data that Uber had gathered with the growth of their data team, Uber didn’t just cut $135 million in wasteful ad spend. It reinvested it where it could fuel explosive growth. Uber Eats was a prime target. Uber leveraged its booming expansion and built-in rider base to increase adoption.

New driver acquisition also took center stage. With demand surging and supply constantly shifting, Uber poured resources into recruiting and retaining drivers to keep the network strong.

Uber also went global, doubling down on Latin America, EMEA and APAC — markets ripe for expansion. 

By shifting budget from underperforming ads to high-impact opportunities, Uber turned a marketing shake-up into a massive growth accelerator.

Key takeaways from Uber’s testing approach

Uber’s measurement approach gives us powerful lessons in the importance of testing. The key insights?

You can’t just assume your ads are working

Uber saved $35 million by running an incrementality test that showed Meta ads weren’t actually bringing in new riders. Without testing, they would’ve kept spending on something that wasn’t driving real growth, not only losing the invested budget but the opportunity cost of not investing recovered spend in growth drivers like Uber Eats and global expansion. 

If you’re not measuring the true impact of your marketing spend, you’re not only wasting money but missing out on opportunities you may not even be aware of yet.

Ad fraud is a bigger problem than you think

Uber lost $100 million to ad fraud — money that was going straight to bots and fake clicks. It took cutting a massive amount of spend before they realized what was happening. If you’re not actively looking for fraud and verifying where your ad dollars go, you could be throwing away a huge chunk of your budget.

Your marketing strategy needs to change as your business grows

In 2018, Uber realized it had already acquired most potential riders in key cities. Continuing to spend aggressively on customer acquisition didn’t make sense anymore. A lot of companies miss insights like this because they don’t have the data they need to uncover them. Remember, what worked when you were scaling fast might not work once you’re established. 

If your marketing spend isn’t evolving, you’re likely overspending in areas that no longer need it and missing opportunities for investment in more profitable areas.

A data-driven culture is everything

In Sundar’s own words, Uber isn’t data-driven; it’s data-obsessed. Uber was able to make fast, confident decisions because it invested in building a data culture from the start. These are just a few of the areas they doubled down on: 

• Uber hired top analytics talent who were not only able to implement advanced measurement techniques but also knew how to interpret the results to make recommendations.
• They invested in their measurement systems to make sure insights were reliable and could be trusted for important decisions. 
• They listened to the data. Ask any data scientist. The greatest challenge they face is not gathering the insights. It’s getting their audience to listen to the recommendations they make, then take action. Uber valued the insights they were given and used them to change their business reality for the better. 

Why incrementality testing matters for business growth

With the attribution challenges and declining access to third-party data, incrementality testing is critical for businesses that want to spend their marketing dollars more wisely.

Beyond saving money, why else does incrementality testing matter more now than ever?

Incrementality testing separates real growth from vanity metrics

Many marketers assume their ads drive conversions when customers may have purchased anyway. Incrementality testing ensures you measure true incremental lift rather than over-attributing success to paid channels that aren’t actually converting.

It enables data-backed budget decisions

Uber’s incrementality test wasn’t just about cutting costs. It was about knowing exactly where money was working, allowing marketing leaders to allocate spend efficiently and drive measurable growth. 

And they aren’t the only ones. A major beauty brand conducting an incrementality test for Performance Max discovered an incremental ROAS of £6, indicating that every £1 spent resulted in £6 of additional revenue.

That’s a 600% return on ad spend, which showed them where to invest more and where to pull back spending.

Incrementality testing helps marketing prove its value to finance

CMOs often struggle to justify budgets when traditional attribution models fail to capture long-term impact. Incrementality testing provides clear data on how campaigns contribute to revenue, making CFO conversations more strategic.

It protects against market shifts and ad fraud

Digital advertising is constantly changing with privacy regulations, algorithm updates and fraud risks, making traditional measurement unreliable. Incrementality testing helps you adapt quickly and invest where it matters.

Incrementality testing turns marketing into a predictable growth driver

Marketing budgets should fuel measurable business outcomes, not just assumed impact. Incrementality testing ensures that every dollar spent is contributing to real and sustainable business growth.

Make every dollar count with incrementality testing 

Incrementality testing is a necessity for marketers who want to make the most of their budgets. In a world where marketing dollars can disappear into non-incremental campaigns, blindly trusting traditional attribution models isn’t enough. If you’re not testing, you’re guessing, and that guesswork can lead to millions in wasted spend.

By adopting a structured approach to incrementality testing you can ensure every dollar is working toward real business growth. Whether it’s identifying underperforming channels, reallocating budget to higher-impact strategies or proving marketing’s true value to finance teams, the insights gained from these tests can transform the way you approach spending decisions.

For marketers looking to be more budget-conscious without sacrificing performance, incrementality testing is the key to making smarter data-driven investments.

To dive deeper into this topic, hear firsthand how Sundar saved Uber $35 million using these strategies.