Why do marketers care so much about incrementality right now?

Thomas Wan: Most mobile teams still lean on MMP attribution models like last-click to decide which ad “caused’’ a conversion. Attribution only tells you who got credit. It does not tell you if the ad changed the outcome.

In practice, ads are often shown to people who could have converted anyway (i.e., organic traffic). That spend does not create lift. Incrementality measurement closes that gap by estimating what would have happened without the ad and isolating the true impact of paid media.

It's popular because it maps to business outcomes, is easy to explain to non-marketers, and gives CFOs what they want most: proof of real ROI, not credit assignment.

‍

What is PSA testing, and how is it structured technically?

Thomas: PSA testing is one way to measure incrementality. At a high level we split users into two groups and compare outcomes:

• Test group: sees the advertiser’s real ads
  
• Control group: sees PSA creative (brand-safe house or Bidease branding)
  

At Bidease, we keep everything else constant so the only difference is the ad itself. Here's a step-by-step look at our tests:

1. We assign users to groups by device ID
   
2. The test group sees client ads; the control group sees Bidease brand PSAs
   
3. We continue until we reach statistical significance
   
4. The difference in outcomes (e.g., installs) between test and control is the incremental lift
   
5. Our normal ML targeting and bidding operate for both groups, so we compare like with like

Where, exactly, does PSA incrementality measure impact?

Thomas: We measure impact at the creative level. The only difference between groups is whether they saw the advertiser’s creative or a PSA ad. While you can scope a test to a single exchange, geo, or placement to study lift within that slice, the question remains, "Did the user see a client or PSA ad creative?” Be clear up front about the decision you want the test to inform.
‍

There are other ways to test incrementality, right? For example, what are ghost ads, and how do they compare to PSA?

Thomas: Ghost ads are an incrementality testing style that also splits audiences by device ID, but the control users do not receive an actual impression. In theory, ghost ads are cheaper and more efficient because you don't pay for control impressions.

However, it comes with operational challenges: you need deep integrations with supply partners to reliably label “would-have-seen’’ users in the control group. Visibility is limited, auditing is difficult, and you are trusting partners to measure themselves. In many open-internet scenarios we cannot enable ghost ads because we do not get the auction-level data needed to tag the control group correctly.

The bottom line is that ghost ads can be ideal where supported and auditable. However, PSA is currently the most practical and transparent option across the broader programmatic ecosystem.

‍

Is PSA a complement or a replacement for ghost testing?

Thomas: Think of PSA as the default and ghost ads as the complement when the environment supports them. PSA works almost everywhere and is fully auditable. If you can get a trustworthy ghost-ad setup, use it to validate and scale. Most marketers will get farther, faster with PSA.
‍

I know many teams want to run incrementality tests, but aren't always able to. What are the biggest challenges teams face when defining and running incrementality?

Thomas: There are five challenges that come up over and over:

1. Defining the Right Counterfactual
   At the heart of incrementality is what we call the “counterfactual,” or, what would have happened if the ad had never been shown. A lot of advertisers think of this as ads versus no ads at all, but that isn’t realistic. Instead, the question should be: what would these users have done if you had not spent money in that specific channel or campaign?
   
   This distinction is important. Attribution models like last-click might show that an ad caused a conversion, but it’s quite possible the user would have converted anyway through organic search, word of mouth, or browsing the app store. So last-click shows attribution lift, not true causal lift. The challenge is building tests that can tell the difference between conversions the ad actually caused and conversions that would have happened regardless.
   

2. Choosing a Feasible Testing Method
   There’s no one-size-fits-all framework. Ghost ads can be cost-efficient but usually require walled-garden integrations (Meta, Google), which limits availability and transparency. PSA tests work universally across exchanges, but they require extra spend and longer timelines. Geo-experiments can run at scale, but they’re noisy because of external market effects. Most advertisers default to the easiest option instead of the most accurate, which creates trade-offs in validity.
   
   
3. Balancing Statistical Power and Budget
   Incremental lift is often small. Sometimes it's only a few percentage points. Detecting these small effects requires large sample sizes, which means more impressions, longer test windows, and higher costs. Many teams underinvest, end tests too early, or spread budget too thin, resulting in inconclusive results. The risk is assuming “no incrementality” when the test simply wasn’t powered to detect a modest but meaningful lift.
   
   
4. Internal Alignment on Metrics
   Even when the test design is sound, teams often disagree on what "success" looks like. Marketing may want to measure installs, finance may focus on revenue or ROAS, while product might care about retention or LTV. Without one north-star KPI agreed upon beforehand, results can get dismissed, misinterpreted, or ignored. The challenge isn’t just running the test, but aligning stakeholders on what incrementality success means.
   
   
5. Spillover and Cross-Channel Effects
   Ads in one channel often affect outcomes in another. For example, Facebook ads can lift branded search traffic, or a TV campaign can boost mobile installs. If incrementality is measured narrowly within just one channel, those halo effects can either be missed or, worse, double-counted. The challenge is deciding whether to run tests at a channel level or to design frameworks that account for cross-channel interactions.
   

That sounds like a lot. How do you get statistical significance without huge spend or long windows, then?

Thomas: At Bidease, we tailor the design to your risk and timeline. Some examples include: 

• Using a non-even split (ex. 90/10) instead of 50/50 to conserve spend while gathering control data
  
• Capping the test duration (for example, two weeks) and accept the power trade-off
  
• Limiting geos or inventory to concentrate signal
  

We make test design choices a collaborative experience. Typically, we recommend the smallest test that still has enough power to answer your incrementality questions.

‍

How do SKAN and other privacy frameworks complicate incrementality?

Thomas: Privacy thresholds, delays, aggregation and inability to split users by device ID make true lift measurement with SKAN or AAK very difficult at a reasonable cost. You often need large budgets and long windows just to meet the data-release thresholds, and even then the data is delayed and coarse. Practically, teams should expect lower fidelity and plan tests accordingly.

‍

Looking ahead, where is incrementality heading in the next 1–2 years?

Thomas: Privacy will keep raising the bar on data access. Expect more opacity, coarser data, and higher costs to reach significance. The opportunity is to standardize pragmatic designs like PSA, pre-align on metrics, and wire lift into your budgeting logic now so the learning compounds over time.

‍

If you could give every app marketer one tip before their first test, what would it be?

Thomas: Align on the five challenges above, especially the counterfactual and the north-star KPI. With those pieces in place, incrementality testing can shift from a one-off experiment into a repeatable process that your marketing, product, and finance teams can rally around.

‍

At Bidease, we’ve built PSA incrementality testing into our toolkit to give marketers a clear, reliable view of campaign impact. Let’s explore how we can put this framework to work for your app. Get started today for a free consultation with a Bidease growth expert. 

‍

Heading 1

Heading 2

Heading 3

Heading 4

Heading 5

Heading 6

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua. Ut enim ad minim veniam, quis nostrud exercitation ullamco laboris nisi ut aliquip ex ea commodo consequat. Duis aute irure dolor in reprehenderit in voluptate velit esse cillum dolore eu fugiat nulla pariatur.

Block quote

Ordered list

1. Item 1
2. Item 2
3. Item 3

Unordered list

• Item A
• Item B
• Item C

Text link

Bold text

Emphasis

^Superscript

_Subscript