MMM vs. Incrementality Testing vs. Attribution: How They Differ and When to Use Each

Three marketers walk into a meeting. One says Facebook ROAS is 4.2x. One says the incrementality test showed Facebook drove 40% lift. One says the MMM shows Facebook has negative ROI. They're all looking at the same channel, the same time period. Who's right?

The answer: they're measuring different things. Understanding why is the key to making sense of conflicting measurement data, and to knowing which number to act on.

What each method is actually doing

Before comparing them, it helps to be precise about what each method measures, because the terms get used interchangeably in ways that create real confusion.

The most important column is "Causal?" Incrementality testing is the only method that actually proves whether your advertising is causing conversions. MMM gets close by controlling for external factors. Attribution doesn't try, it assigns credit based on presence, not causality.

Media mix modeling: what it's for

MMM takes your historical data, weekly ad spend by channel, plus external factors like seasonality, promotions, price changes, and economic conditions, and builds a statistical model that decomposes how much of your sales came from each source. The output is something like: "Of your $2M in monthly revenue, roughly $400k came from paid search, $250k from Facebook, $180k from YouTube, and $1.17M would have happened without any advertising."

Use MMM when:

• You need to make portfolio-level budget allocation decisions (shift $500k from Facebook to YouTube, is it worth it?)
• You're spending on brand and reach channels (TV, audio, OOH) that don't show up in attribution
• You're planning future scenarios ("what happens to revenue if we cut total spend by 20%?")
• You want a measurement approach that doesn't depend on cookies or user-level tracking

Don't use MMM when:

• You need answers in days or weeks, a properly calibrated model takes months to build
• Your annual ad spend is under ~$500k (too little signal to detect effects reliably)
• You want to understand individual campaign or creative performance, MMM operates at channel level, not campaign level

Incrementality testing: what it's for

Incrementality tests answer a single, sharp question: did this advertising cause these conversions, or would they have happened anyway? You do this by creating a control group, people who are withheld from seeing your ads, and comparing their behavior to the group that saw the ads. The difference in conversion rate is your incremental lift.

The most common formats are holdout tests (audience-level suppression within a platform), geo experiments (dark markets vs. live markets), and ghost ads (showing PSA ads to the control group to equalize ad exposure effects).

Use incrementality testing when:

• You suspect a channel's reported ROAS is inflated, especially retargeting, branded search, and any channel where you're reaching high-intent audiences
• You want to prove causality before making a major budget change
• You're calibrating your MMM, incrementality results can serve as priors or constraints in a Bayesian model
• You want to distinguish prospecting from retargeting efficiency on the same channel

Don't use incrementality testing when:

• You want insights faster than 4 weeks, an underpowered short test is worse than no test
• You want to measure all channels simultaneously, each test covers one channel or campaign type
• Your holdout group would meaningfully reduce reach in a way that hurts short-term performance

Attribution: what it's for

Attribution tracks the digital touchpoints a user encounters before converting and distributes conversion credit among them. Last-click gives all credit to the final touchpoint. Data-driven distributes credit based on statistical patterns across observed conversion paths. Multi-touch splits credit according to a rule (linear, time decay, position-based).

Attribution is fast, granular, and built into every major ad platform. It's also structurally limited in ways that matter enormously for decision-making.

Use attribution when:

• Comparing campaigns within a single platform, Facebook's attribution data tells you which Facebook audiences and creatives are outperforming others
• Understanding journey sequences, what channels appear first vs. last in high-value customer paths?
• Operating at a speed that MMM and incrementality can't match, optimizing creative, bidding, and targeting

Don't use attribution when:

• Comparing performance across channels, each platform's attribution is self-reported and will credit itself too generously
• Making major budget allocation decisions, attributed ROAS reflects correlation, not causality
• You have significant tracking gaps (which you almost certainly do, especially post-iOS 14.5)

How they work together

The right mental model: MMM handles strategy, incrementality handles validation, attribution handles operations.

MMM tells you where your budget should go at a portfolio level. It's slow, expensive, and right about broad directional signals. Attribution tells you which campaigns and creatives to run within each channel. It's fast, cheap, and systematically biased toward bottom-funnel. Incrementality testing sits between them, it validates that the channels your MMM favors are actually driving incremental lift, and it calibrates the conversion inflation that attribution produces.

For the three-marketer scenario at the top: the MMM result (negative ROI) might reflect that Facebook's aggregate historical spend hasn't correlated with revenue growth after controlling for external factors. The incrementality result (40% lift) might reflect a specific well-designed prospecting test. The attribution result (4.2x ROAS) likely includes a lot of organic conversions that Facebook is claiming credit for. None of them is wrong, they're measuring different things over different windows with different methodologies.

The recommended sequencing

Most companies can't implement all three measurement approaches at once. Here's the order that makes sense:

1. Attribution first, you probably already have it. Use it for within-channel optimization, not cross-channel comparison.
2. Incrementality second, start with your highest-spend channels and your highest-suspicion channels (usually retargeting). Run a holdout test. The results will be clarifying.
3. MMM third, when your total annual ad spend exceeds roughly $1M and you need a model for strategic budget allocation. Use incrementality results to calibrate it.

This is the sequencing logic behind marketing triangulation: build the measurement stack iteratively, use each method to check and improve the others, and don't make strategic decisions based on attribution data alone.