Which of your ads actually cause sales?
Platform dashboards report the sales they can see near an ad. They can't tell you which sales would have happened anyway. Matched market tests can. We design, run and read geo incrementality tests so you know your true incremental ROAS, channel by channel.
Attribution counts correlation. Budgets need causation.
Retargeting, branded search and broad social campaigns often look brilliant in platform reports because they reach people who were already going to buy.
That isn't a theory. When eBay switched off its branded search ads in a large controlled experiment, almost all of the traffic simply arrived through organic results instead (Blake, Nosko & Tadelis, Econometrica, 2015 (opens in a new tab)). Across 15 large Facebook field experiments, common observational methods frequently failed to recover the true effect measured by randomized tests (Gordon et al., Marketing Science, 2019 (opens in a new tab)).
An incrementality test asks one clean question: what happened in places that saw the ads, compared with similar places that didn't?
Good reasons to run a test
- You are about to scale a channel and want to know if the next dollar pays back
- Finance doesn't trust the ROAS in ad platform dashboards
- You suspect branded search or retargeting is taking credit for organic demand
- A new channel (CTV, YouTube, podcasts, out-of-home) has no click trail to attribute
- Platform and analytics numbers disagree and someone has to break the tie
Hold out regions. Compare. Measure the lift.
Geo experiments work with aggregated sales by region, so they don't depend on cookies, device IDs or click tracking. Google researchers described the approach in 2011, and open-source tools such as Meta's GeoLift and Google's Trimmed Match now make it practical for mid-sized advertisers.
Design
Choose the question, the KPI and the geography (provinces, census metropolitan areas or postal-code regions). Use historical data to pick test and control markets that move together, and run a power analysis to size the test.
Run
Change spend only in test markets: turn a channel on, off or up. Keep everything else steady. Typical tests run four to eight weeks, plus a cool-down to capture lagged sales.
Read
Compare actual results in test markets with the counterfactual built from controls. Report incremental conversions, incremental revenue, iROAS and a confidence interval.
Reallocate
Turn the result into a budget decision and a calibration factor for your day-to-day attribution, then plan the next test.
The right test for the question.
Geo holdout
Pause a channel in a set of matched regions to see what it really adds. Best for retargeting, branded search and always-on campaigns.
Geo scale-up
Increase spend in test regions to find out whether the next dollar still pays. Best before a big budget increase.
Platform lift studies
Meta Conversion Lift and Google Ads Conversion Lift, set up and read independently, with results reconciled against your own sales data.
A decision, not just a deck.
- Test design document: hypothesis, KPI, markets, duration, budget and minimum detectable effect
- Market selection and power analysis on your historical data
- Campaign set-up checklist for Google Ads, Meta and other channels
- Mid-test monitoring for contamination and spend drift
- Readout: incremental conversions, revenue, iROAS, cost per incremental acquisition and confidence interval
- Budget recommendation and attribution calibration factors
- Reusable data pipeline and code, handed over to you
What we need from you
- Daily or weekly sales, leads or sign-ups by region for at least the past several months (a year or more is better)
- Spend by channel and region for the same period
- A channel or budget change you are genuinely willing to make in test markets
- A promotions calendar, so we don't test through a major sale by accident
Not sure your data is ready? The first conversation is a free feasibility check.
Incrementality testing questions.
What is incrementality testing?
Incrementality testing measures the extra conversions or revenue a marketing activity causes, compared with what would have happened without it. It uses a control group, such as matched regions that don't see the ads, to estimate that counterfactual.
What is a matched market test?
A matched market (or geo) test compares regions that receive a marketing change with similar regions that don't. Markets are matched on historical sales patterns so that any difference during the test can be attributed to the change.
Can geo tests work in Canada, with fewer large markets than the US?
Yes, with care. Canada has a small number of very large metros, so we often use provinces, census metropolitan areas and smaller regions built from postal-code areas, combine several markets per group, and use synthetic control or trimmed match methods that cope with uneven market sizes. Toronto, Montréal and Vancouver are usually kept in control or handled separately.
How long does a test take?
Most run four to eight weeks of live testing after one to two weeks of design, plus a short post-period to catch delayed purchases. Longer purchase cycles need longer tests.
How much spend do we need?
It depends on how noisy your sales are and how big an effect you need to detect. The power analysis tells us before you commit. If the test can't detect a realistic effect with your budget, we'll say so and suggest a different design.
What is iROAS?
Incremental return on ad spend: incremental revenue caused by the ads divided by the ad spend. Unlike platform-reported ROAS, it excludes sales that would have happened anyway.
Do we lose sales by holding out regions?
Sometimes a little, in the holdout regions, for the test period. That's the cost of knowing. Scale-up tests avoid holdouts entirely, and well-designed tests keep the holdout as small as the statistics allow.
Find out what your ads really add.
Tell us which channel you'd most like to prove or disprove. We'll check whether your data can support a clean test and sketch the design.