Demand Spring

Incrementality Testing

Incrementality Testing is an experimental method that measures the true added impact of a marketing activity by comparing exposed and unexposed groups to isolate causal effect.

Also known as: lift testing, incremental lift testing, causal measurement

Incrementality Testing is a measurement approach that isolates the genuine, additional results a marketing activity produces. It compares a group exposed to a campaign against a control group that was not, to see what would have happened anyway without the marketing intervention. It is the discipline that separates correlation from causation in attribution.

What Incrementality Testing Means

Incrementality Testing uses controlled experiments such as holdout tests, geographic splits, or PSA ad-replacement tests to measure the lift attributable to a marketing intervention. The output is incremental conversions, incremental revenue, and incremental ROAS. Incremental ROAS is almost always lower than standard ROAS, often by 30 to 60 percent on retargeting and bottom-funnel campaigns. The gap reveals how much of standard ROAS is conversions that would have happened anyway, which is the question CFOs actually want answered when they ask whether marketing spend works.

How Incrementality Testing Works

Comparable test and control groups are created through random assignment or matched-market design, only the test group is exposed to the activity, and the difference in outcomes estimates true incremental impact. Pre-registering the hypothesis, sample size, and analysis plan avoids the temptation to slice the data after the fact. Small or biased samples produce unreliable conclusions, so the method needs careful experiment design and enough volume to reach statistical confidence. It also needs the discipline to suppress marketing to a control group, which is the part most marketing teams resist hardest.

Common Pitfalls and Misconceptions

The practical point is that Incrementality Testing directly addresses the biggest flaw in standard attribution: crediting conversions that would have happened anyway. The biggest pitfall is running the test, finding low or negative lift, and then dismissing the result because it contradicts the attribution dashboard. Trust the experiment, recalibrate the model. The second pitfall is underpowered sample sizes that produce inconclusive results, and the third is contamination between test and control groups when targeting boundaries are leaky, which biases results toward zero lift and undermines the test’s validity.

Incrementality Testing in Practice

The practitioner reality is that Incrementality Testing is expensive in opportunity cost, so it should be reserved for decisions that matter. Don’t run an incrementality test on a 50,000 dollar campaign; do run one on a 5 million dollar always-on program, a brand-spend defense, or a retargeting line item the CFO suspects is mostly converting people who would have converted anyway. The most disciplined revenue orgs build an incrementality calendar that covers their largest spend lines once every 12 to 18 months, then use those validated lift figures to calibrate attribution models in between tests. This dual-system approach is the modern measurement standard.

Back to the Glossary

Common questions.

Why is incrementality testing important?
Standard attribution often credits conversions that would have happened anyway, particularly in bottom-funnel and retargeting channels where buyers were already going to convert. Incrementality testing measures only the additional results truly caused by marketing, which is the question finance and leadership actually want answered.
How do you run an incrementality test?
Create comparable test and control groups through random assignment or matched-market design, expose only the test group to the activity, and measure the difference in outcomes to estimate true incremental impact. Pre-register the hypothesis, sample size, and analysis plan to avoid the temptation to slice the data after the fact.
What does incrementality testing require?
It needs careful experiment design, comparable groups, and enough volume to reach statistical confidence. Small or biased samples produce unreliable conclusions. It also needs the discipline to suppress marketing to a control group, which is the part most marketing teams resist hardest.
How does incrementality testing differ from attribution?
Attribution assigns credit for conversions that occurred, but it cannot tell whether those conversions would have happened anyway. Incrementality testing uses a control group to isolate the conversions truly caused by the activity. It answers a causal question that attribution alone cannot, which is why investors and CFOs take it seriously.
When is incrementality testing worth doing?
It is most worthwhile for significant, ongoing spend where the question of true impact carries real budget consequences, such as a major advertising channel or always-on program. It is harder to justify for small or one-off activities that lack the volume for statistical confidence. Reserve it for decisions that matter.
What is incremental ROAS?
Incremental ROAS measures revenue caused by an ad campaign (test minus control) divided by ad spend, as opposed to standard ROAS which counts all attributed revenue. Incremental ROAS is almost always lower than standard ROAS, often by 30 to 60 percent on retargeting and bottom-funnel campaigns. The gap reveals how much of standard ROAS is conversions that would have happened anyway.
What are common pitfalls in incrementality testing?
Underpowered sample sizes, contamination between test and control groups, post-hoc data slicing, and testing too short a window to see delayed effects all undermine results. The biggest pitfall is running the test, finding low or negative lift, and then dismissing the result because it contradicts the attribution dashboard. Trust the experiment, recalibrate the model.

Related Terms

More from Measurement.

  • Algorithmic Attribution

    Algorithmic Attribution is a data-driven approach that uses statistical or machine learning models to assign conversion credit based on each touchpoint's measured contribution rather than a fixed rule.

  • Annual Recurring Revenue (ARR)

    Annual Recurring Revenue (ARR) is the value of the recurring components of a subscription business normalized to a one-year period, excluding one-time fees.

  • Attribution Window

    Attribution Window is the defined time period during which a marketing touchpoint can be credited for a resulting conversion in an attribution model.

  • Benchmarking

    Benchmarking is the practice of comparing performance metrics against past results, competitors, or industry standards to assess how performance compares to a reference point.

  • Bottom-Up Forecasting

    Bottom-Up Forecasting is a forecasting method that builds revenue projections by summing individual deals, accounts, or program estimates from the ground up rather than dividing a top-line target downward.

  • Bounce Rate

    Bounce Rate is the percentage of website sessions in which a visitor views a single page and leaves without further interaction or navigating to another page.

Back to the Glossary Talk to a Strategist

Let’s Talk

Let’s talk about what your next quarter could look like.

Tell us what you’re working on. A senior practitioner reads it, not an SDR queue, and replies, usually within one business day.

  • Reviewed personally, not routed through a queue.
  • A conversation about what you’re actually working on, not a generic pitch.
  • No pressure, just a chance to talk it through.

Book a conversation