Consent and tracking loss

Your analytics platform is not measuring your traffic. It’s measuring the subset of your traffic that consented, wasn’t blocked, and whose identifiers survived long enough to be joined up. On a UK or EU site that subset can be well under the full population.

The loss itself is a nuisance. The bias is the actual problem, and the two get conflated constantly.

Where the traffic goes

• Consent. Users who reject non-essential cookies disappear from analytics entirely. Opt-in rates vary enormously by region and by banner design, and the banner is itself a conversion surface that almost nobody tests.
• Browser restrictions. Safari’s tracking prevention caps the lifetime of client-set cookies to a few days, and to 24 hours in some cases. Firefox blocks known trackers outright. The practical effect is that returning-visitor recognition decays fast, so your “new” visitor count is inflated and any window longer than a week is unreliable.
• Ad and script blockers. A meaningful slice of traffic, and heavily skewed - more technical, more desktop, different demographics.
• Server-rendered and app contexts where the tag never fires at all.

Why the bias matters more than the volume

If you lost 30% of your data at random you’d simply have less data, wider intervals, and no other consequence. You don’t lose it at random. Blocked and non-consenting users differ systematically in device, technical sophistication, privacy attitude, and often in purchase behaviour.

So every absolute number you report is an estimate of a population you didn’t sample. Your measured conversion rate is the conversion rate of the observable subset, not of your visitors. Anyone reporting site conversion rate to two decimal places without knowing their observation rate is reporting a number with an unknown offset.

The first thing to do is size your own gap: compare orders recorded in your analytics against orders in your commerce backend for the same period. That ratio is your observation rate, it’s usually worse than people expect, and it should be attached to every absolute figure you circulate.

What this does to experiments

Better news here, and it’s the important distinction.

Experiments measure a relative difference between two groups drawn from the same population. If tracking loss is balanced across variants, the unobserved users are missing symmetrically, randomisation still holds among those you do see, and the relative lift is roughly unbiased even though the absolute rates are wrong. You can trust the direction and approximate size of the effect while distrusting the levels.

The exception is differential loss, where a variant changes who gets tracked. A treatment that loads an extra script a blocker recognises, or that shifts the consent banner, or that renders slower so more users leave before the tag fires - now the missingness correlates with assignment and the comparison is contaminated. This is missing not at random arriving through the front door.

It usually announces itself as sample ratio mismatch. That’s the check that catches it, which is one more reason SRM deserves to be automatic rather than something you remember to look at.

Modelled conversions

Ad platforms and analytics tools fill the gap with modelling - estimating the conversions they couldn’t observe from the ones they could. Google’s consent mode behaviour is the widely deployed version.

For ad platform optimisation this is genuinely useful. For experiment analysis it’s dangerous, and the reason is simple: the model has no idea your test exists. It’s imputing conversions from historical patterns that pre-date your variant, so the modelled portion of your data is pulled toward the status quo. Analysing a test on data containing modelled conversions dilutes real effects toward zero.

Never read a test off modelled data. Use the observed subset, or use server-side collection.

What to do about it

• Measure your observation rate against backend truth, and quote it alongside absolute metrics.
• Treat the consent banner as a CRO surface. Wording, layout and default placement move opt-in rates a lot, and every point of opt-in is measurement capacity you get back. Stay the right side of the line - a banner engineered to trick consent is both a legal problem and, given the reject rate that follows a nagging banner, often self-defeating.
• Move decision-critical measurement server-side, where the collection doesn’t depend on the browser cooperating. That’s the subject of server-side tagging and conversion APIs.
• Expect identity stitching to degrade over time rather than hold, and design retention and repeat-purchase reporting around backend identity where you have it.
• Read attribution output with the loss rate in mind. The channels that lose most to blocking look worse than they are, which quietly biases budget away from them.