How to evaluate national TV when you can't geo-test it

Last updated: September 17, 2026

National TV is a channel where marketers want a clean read on incrementality, but in some situations they cannot run a standard geo holdout. Delivery is often national by default, spend is lumpy, and the same campaign can affect branded search, direct traffic, and assisted conversions in ways that do not show up cleanly in short windows. That does not mean TV is unmeasurable. It means the test design has to find another source of variation to substitute for the geographic split you can't run, and the interpretation needs to account for that substitution.

Why can some advertisers geo-test linear TV when others can't?

This is worth answering before jumping to workarounds, because it isn't really about "linear vs. CTV" — it's about how the buy itself is structured.

Some TV buys give the advertiser control over which markets receive the ad: specific station or spot buys, or campaigns deliberately split between two comparable regions. That control is what makes a clean geo experiment possible in the first place — you need a lever to hold one market back as a control while the campaign runs elsewhere.

Other buys are placed at the rotation or host level — inventory runs across a mix of programs, dayparts, or stations without market-by-market insertion control, often because that's the more efficient or more accessible way to secure the reach an advertiser wants. When TV is bought this way, there's no lever to isolate one market from another, so a standard geo holdout isn't on the table no matter how much the advertiser wants one. The same constraint tends to show up for direct-response-style TV bought nationally, where inventory is priced and placed opportunistically rather than market by market.

Even among advertisers who can geo-target TV, the market unit matters. DMA-level splits are usually the better fit for linear TV, since that's the level at which inventory is typically planned and delivered — station coverage areas map much more closely to DMAs than to state lines. A state-level design is sometimes workable when the surrounding media plan and reporting can be aligned to state boundaries, but it's the less common case for TV specifically.

In practice, this creates a genuine split: some advertisers can run a clean geo test on their linear TV, and others structurally can't, even within otherwise similar buys. The practical takeaway is to ask specifically whether a given buy is structured at the station/DMA level or the rotation/host level — that answer predicts geo-testability far better than the channel label does.

When a DMA-level split is actually worth running

Buy structure gets you in the door, but two more things need to be true before a DMA holdout is actually the right call:

The primary metric needs enough volume by geography. A market split only works if the outcome you're measuring happens often enough within each DMA to detect a real difference. A metric that's sparse at the market level can make even a well-structured geo test unreadable, regardless of how cleanly the media can be split.
Spillover risk needs to be manageable. TV is broad-reach media, and adjacent-market spillover or brand halo can blur the line between "treatment" and "control" markets. That's not a reason to avoid a geo test, but it does need to be accounted for in market selection and interpretation — markets right next to each other, or served by overlapping stations, are weaker candidates for a clean split.

When buy structure, metric volume, and spillover risk all check out, a DMA-level test can be designed the same way as other geo experiments: markets that tracked similarly in the pre-period, one held steady as control, the other changed, and the divergence during the test window measured. When any of the three doesn't hold, that's a signal to look at the alternatives below rather than force a weak geo split.

If you cannot geo-target TV, what kind of test is still possible?

Most of the options below are doing the same underlying thing: creating or finding enough variation in TV spend over time so that its relationship to your outcome metric becomes visible. A geo test gets its evidence from variation across markets at a single point in time; without that lever, the main source of variation left is time — spend has to move enough, and cleanly enough, for a model or a simple comparison to pick up its effect on the metric that matters.

That reframes the question from "which single test should I run" to "how do I create or find the cleanest, largest spend variation I can, given what my media plan and risk tolerance allow." The options sit on a spectrum:

A full go-dark creates the largest possible variation — spend goes from its normal level all the way to zero — which tends to produce the clearest signal per unit of time, at the cost of the most business risk and, often, real operational and pricing costs from pausing entirely.
A pulsed or switchback design, or a brand's existing flight pattern, creates repeated, smaller swings between on and off. Each individual swing carries a weaker signal than a full go-dark, but repeating it several times helps average out noise and spreads the risk across periods rather than concentrating it in one.
A spend-variance (dose-response) test never turns TV off at all — it just needs the spend range to be wide enough (at least 2x between high and low periods) and sustained long enough (three to four months at minimum) for the relationship to surface.
Opportunistic or partial geo variation, where it exists, substitutes cross-market variation for time variation, even without a formally designed split.

Whichever option you use, the variance only tells you something if you can plausibly attribute it to TV alone. A large spend swing that coincides with a promotion, a pricing change, or a competitor's move is not useful evidence — the size of the variation matters less than whether it's clean enough to rule out other explanations. That's the reason behind guidance like keeping the rest of the mix stable during a go-dark, or checking whether a brand's historical flights are confounded with seasonality before trusting them: it's less about the test design itself and more about whether the variation it produces is attributable.

Two options sit outside this spectrum, for different reasons. Survey-based measurement (a "how did you hear about us" survey, or a brand lift study) measures awareness and recall directly rather than inferring an effect from how spend and outcomes move together — worth running alongside a variance-based method rather than instead of one, since it's a different kind of evidence, not a substitute. Borrowing a geo test from a channel that can actually run one — typically connected or addressable TV — is still a real experiment with real variance, just variance in a different channel's delivery, not linear TV's; the result has to be carried over to linear by judgment about audience and delivery differences, which is why it belongs alongside the alternatives below rather than as a like-for-like substitute.

A full go-dark: the maximum-variance version

For TV that runs continuously, a planned go-dark is usually the most direct way to generate a contrast that wouldn't otherwise exist:

Pick the decision first. The question should be concrete: "Should we keep funding national TV at the current level?" is better than "Is TV good?"
Choose the primary KPI before launch. Use the downstream metric that matters for the budget decision: purchases, qualified leads, bookings, subscriptions, or revenue. Avoid switching metrics after the read comes back.
Choose a window where the business is as stable as possible. Avoid major promotions, assortment changes, pricing changes, site migrations, or unusual seasonality if you can.
Run a true blackout. If the design is meant to test TV, pause TV fully for the agreed period rather than reducing it halfway and hoping the signal is still interpretable.
Keep the rest of the mix as stable as possible. If search, paid social, email, or affiliate activity changes materially during the blackout, the read becomes much harder to trust.
Pre-define guardrails. Before launch, agree on the business thresholds that would trigger an early stop if the downside is too large.
Compare before, during, and after — ideally against a synthetic control, not just your own history. The strongest read doesn't just compare the blackout window to your own pre-period trend; it builds a counterfactual from comparable series — other markets, products, or correlated metrics that didn't see the pause — and measures the gap between what actually happened and what that counterfactual predicts. A pulse large enough produces a gap that's legible on its own, without needing a heavy statistical model to interpret it. If the KPI dips during the go-dark and rebounds after TV returns, relative to that counterfactual, confidence rises.

Decide the counterfactual before the flight — a pulse nobody planned to measure is just spend.

One planning detail worth building in upfront: for upper-funnel channels like TV, the readout usually needs a cooldown period after the blackout window ends before you finalize the analysis. Conversions influenced by pre-pause exposure can continue to land for days or weeks afterward, and closing the books too quickly can understate, or misattribute, the effect.

What this design can and cannot tell you: it can tell you whether turning TV off appears to change the business meaningfully enough to affect the budget decision. It cannot tell you cleanly the exact channel-level causal lift with the same precision as a strong geo experiment. It's usually best used directionally for a large budget question — keep funding, reduce, or re-test under better conditions — rather than for a precise lift number.

If TV already runs in pulses, look at what you already have before creating new variance

A full go-dark's value comes specifically from creating variation that doesn't already exist. If a brand already flights its TV — running in bursts and going quiet between them as a matter of normal planning — that variation already exists in the historical data, often across many cycles.

In that case, the more useful first move is usually to analyze the flights you already have rather than schedule a new pause: treat each historical on-period and off-period as its own data point and look for a consistent relationship between TV weight and the outcome metric. This has a real advantage over a fresh go-dark — you don't have to wait for a new test window, because the variation you need already exists in your history.

The caveat is that existing flights are only as trustworthy as their independence from everything else that was changing at the same time. Pulses are usually timed for business reasons — budget cycles, seasonality, launches — rather than measurement reasons, so "off" periods often coincide with other shifts too. If a look at historical flights shows the off-periods are entangled with seasonal or promotional patterns, that history isn't clean enough to lean on by itself, and it's worth layering in one or two deliberately placed pauses — chosen specifically to avoid those confounds — rather than assuming any old flight will do.

Other ways to create the variance you need

If a full go-dark isn't the right fit — the business can't tolerate a complete pause, or the historical flight pattern isn't clean enough to rely on — a few other approaches are worth considering:

A spend-variance (dose-response) test. Let spend vary meaningfully — at least a 2x range between high and low periods — while holding the rest of the mix as stable as you can, over a long enough window (at minimum three to four months) for the relationship between spend and outcome to show up. This is the lowest-risk and usually cheapest option on the spectrum, and once it's running, the evidence compounds on its own — every MMM refresh gets another read, as long as the variation keeps up. The catch cuts the same way in reverse: flat spend teaches the model nothing, so if the range is too narrow or too short, this produces no more insight than the MMM already had.
Opportunistic or partial geo variation. Even when a formal geo split isn't available, some buys still have enough natural variation across markets — uneven weight, staggered rollout timing, a market dropped for inventory reasons — that a market-level comparison can be attempted after the fact. It's not as clean as a planned holdout, but it's a source of evidence that's often overlooked.
Survey-based measurement. As noted above, this doesn't rely on spend variance at all — it measures awareness and recall directly — which makes it a useful independent check on whatever a variance-based method finds, rather than a replacement for one.
A lightweight, ad-hoc directional analysis. Comparing before/after patterns around any planned or unplanned change in TV weight — even without a formal test design — is often close to a one-day exercise that captures much of the signal a heavier analysis would.
Revisiting the buy structure itself. If TV remains a strategic priority, it's worth asking your media partner whether a future buy could be structured to support station/DMA-level control, opening the door to a real geo test — and a source of variation that doesn't depend on time at all — down the line.
Borrowing a geo test from a channel that can run one. If a related channel supports real geo-targeting — connected or addressable TV is the common case — a true randomized holdout there gives you a clean causal read, faster and cheaper than a national pulse on linear TV, and it leaves behind a geo-testing harness other channels can reuse. The catch is that it measures a different audience, not linear TV itself: the finding has to be carried over to linear by judgment, not calculation, and it can't stand in for the remnant or rotation-style delivery that's often exactly why linear couldn't be geo-tested in the first place. Treat it as directional evidence about the broader TV effect, not a substitute number for linear specifically.

Building a counterfactual model earns two different verdicts depending on what it's built from. Paired with a genuinely large, deliberate pulse — the synthetic-control read described in the go-dark section above — it's a legitimate first-tier method: the gap between actual results and a comparable-series counterfactual can be large enough to read directly, without leaning on a fragile model. The real costs are the size of pulse and budget it takes to produce that gap, the assumption that the comparable series is actually a valid twin (it's chosen, not randomized), and the fact that any other simultaneous change — a promotion, a pricing shift — muddies the read just as it would for a plain go-dark.

Building a statistical counterfactual on top of small or passive variation — spend that moved on its own, or barely moved at all, rather than a deliberate pulse — is a different matter, and worth treating as a last resort rather than a first option. It's tempting to reach for something that looks more rigorous than an ad-hoc comparison, but in practice the confidence intervals on these models tend to be wide enough that the added complexity doesn't buy meaningfully more certainty than the simpler version. We've seen advertisers use this route anyway, out of necessity — forecasting what performance would have looked like without the campaign and treating the actual-versus-forecast gap as a working estimate. That's a reasonable stopgap when nothing else is available, but a fallback, not a first choice.

Choosing the right KPI for a TV test

The KPI matters as much as the test design itself, and this applies whether you're running a DMA holdout, a go-dark, or a spend-variance test.

A good KPI for a TV test usually has three traits: it moves quickly enough to show a response within the planned test window, it has enough baseline volume to support a credible comparison, and it still matters to the business as a decision-making outcome. A metric that fails any one of these can make an otherwise well-designed test unreadable.

This is often a real tension. The KPI that matters most for the budget decision — bookings, revenue, subscriptions — is frequently the slowest and sparsest one to move. When that's the case, a faster-moving upstream metric (site visits, qualified leads, a mid-funnel conversion event) can produce a more readable test while still informing the same underlying decision, as long as the link between the upstream metric and the business outcome is well understood going in.

How should you interpret an MMM that assigns little or no incrementality to TV?

Part of the reason MMM and TV tests can disagree is structural, not just a matter of data quality. An MMM is long-horizon and correlational — it looks across many months of spend and outcomes and can pick up lagged and indirect effects, including brand carryover that builds slowly. A TV experiment, by contrast, is short-horizon and causal — it asks whether a specific change in delivery produced a measurable change within the test window. TV in particular tends to have broader, slower-moving effects than lower-funnel channels, so it's not unusual for an MMM to assign it meaningful contribution while a short holdout shows less immediate movement. That's a reason to look more closely at test power, duration, and KPI choice, not necessarily a reason to distrust either number.

A low-TV estimate in the MMM does not automatically mean TV has no effect. It usually means one of three things:

The model genuinely does not see a repeatable relationship between TV variation and the target metric in the historical data.
The TV effect may be hard to detect in the available data because spend did not vary enough, the signal is noisy, or other channels moved alongside TV.
The contribution may be showing up indirectly through lag, halo effects, or portfolio interactions that are harder to isolate cleanly.

This is exactly why TV is a strong candidate for follow-up testing. The right reaction is usually not to declare the MMM wrong or the channel useless. The right reaction is to ask whether the next best evidence should come from creating more spend variation directly, or from a model recalibration informed by new evidence. In fact, reason 2 above is often the whole story: if TV spend never varied enough historically for the model to learn from, a designed test's real job is simply to manufacture the variance the model never had.

What if the spend or outcome totals in the MMM look higher than expected?

When the spend or conversion totals in the model look larger than what a separate internal memo shows, treat that as a data-scope question before a channel-performance question.

Common causes include:

Portfolio or brand-family data being grouped together. A model may include multiple brands, business lines, or related entities if that is how the source data was structured.
Different definitions of the outcome metric. One source may report voucher sales, while another is using a broader conversion or revenue field.
Different time windows. The memo and the model may not be looking at exactly the same dates.
Different preprocessing rules. Deduping, lag treatment, and channel mapping can all change topline totals.

The practical takeaway: if the topline spend or outcome totals do not reconcile, resolve the scope mismatch first. Do not jump straight from a totals mismatch to a conclusion about whether TV is or is not incremental.

Are we training on data that old?

Usually, MMMs are built on a long enough historical window to learn stable relationships across seasonality, promotions, and channel variation. That often means many months or multiple years of data rather than only the most recent quarter.

Using older data is not inherently a problem. It becomes a problem when the older period no longer reflects the current business. The right question is not "Is the data old?" but rather "Is the training window still representative of how the business operates now?"

Older data is more likely to be less representative when there has been a structural break such as:

a major pricing or product change
a new brand or business line entering the portfolio
a major shift in creative strategy or channel mix
a meaningful change in targeting, attribution, or conversion definitions

If a structural break has occurred, the right move is usually to revisit the model scope or training window rather than assume that all history should continue to carry equal weight.

A practical decision framework for national TV

If TV can be geo-targeted, prefer a DMA-level split over a state-level one where possible, since it typically matches how TV inventory is actually planned and delivered, and confirm metric volume and spillover risk before committing to it.
If TV can't be geo-tested under your current buy, check whether restructuring the buy (or working with your media partner) would make station/DMA-level testing possible as that reopens cross-market variation as an option.
If a geo split isn't available, the goal is to create or find spend variation clean enough to attribute to TV alone: a full go-dark for continuous TV, an analysis of existing flights for already-pulsed TV, or a sustained spend-variance test if a full pause isn't tolerable.
When reading a go-dark or a large deliberate pulse, build a synthetic control from comparable series rather than relying on your own pre-period trend alone as it's what makes a big enough gap legible without a fragile model.
If linear TV can't be tested directly and a related channel like CTV supports real geo-targeting, a holdout there can supply faster, cheaper directional evidence, but treat the transfer to linear as judgment, not a substitute number.
Choose a KPI that's fast and reliable enough to move within the test window and still matters for the decision (an upstream proxy is often more useful than a slow, sparse downstream metric).
If the MMM and the test still disagree, investigate data scope, portfolio inclusion, lag, and structural breaks before making a hard budget call.
If the model inputs do not reconcile to internal reporting, fix the scope mismatch first.
If the business has changed materially since the older training period, treat model retraining or recalibration as part of the answer.

For national TV, the goal is usually not false precision. It is to make the next budget decision with the best available evidence, understand the limits of that evidence, and improve the model from there.

A succinct way summary of how to evaluate national TV without a geo-test

Working with your Growth Advisor

None of the designs above need to be decided in isolation. Before committing to a specific test, it's worth confirming with your Growth Advisor whether a DMA-level split is actually viable for your buy, whether a full go-dark or one of the variance-based alternatives is the more realistic option, what KPI is most likely to produce a readable answer, and how the result should be interpreted alongside your MMM once it's in. The goal isn't just to run a test; it's to run one credible enough to change the budget decision it's meant to inform.

Want to learn more? Check out other Paramark experimentation articles here.