Growth Notes

Walled Garden Reporting

Tags: analytics commerce concept

Walled Garden Reporting

Date: 2026-08-16

Every ad platform reports more conversions than you actually had, and the sum exceeds your order count. That’s not a bug to reconcile — it’s four vendors each claiming the same orders under four different models, none of which can see the others.

What it is

A walled garden is a platform that holds its own user identity, serves its own ads, measures its own conversions, and doesn’t share the underlying data. Google, Meta, Amazon, TikTok.

Walled garden reporting is what each one tells you about its performance — computed inside the wall, on data you can’t inspect.

Why the numbers exceed reality

your order system                     1,000 orders

Google Ads reports                      620 conversions
Meta reports                            480
TikTok reports                          210
affiliate platform reports              180
                                      ─────
                                      1,490 claimed

Four causes, all structural:

  • Each claims the same order. A customer who saw a Meta ad, searched on Google and clicked an affiliate link is counted once by each — three claims, one order
  • Different attribution models. Each applies its own to its own data — Attribution Models
  • Different windows. A 7-day click window and a 28-day one credit different sets — Attribution Windows
  • View-Through Attribution. Some count impressions nobody clicked
  • Modelled Conversions. Some of what’s reported was estimated, not observed

In plain terms: each platform is answering “how many of these orders had contact with us?” — and the answers overlap because the customers did. Nobody is double-counting within their own system; the sum is meaningless because the sets intersect.

What you can and can’t do with it

Can:

  • Compare within a platform over time. Same model, same window, same bias — so a change is a real change
  • Compare creative or campaigns inside one platform. The bias applies equally to both sides
  • Read direction. A platform’s reported performance halving means something happened

Cannot:

  • Sum across platforms. The result is not a number
  • Compare platforms to each other. Different models measuring different things
  • Reconcile to your order system. They will never agree, and chasing it is wasted effort
  • Allocate budget between platforms on reported return. This is the expensive mistake — the platform with the most generous attribution wins the budget, regardless of contribution

What to use instead for allocation

  • Your own attribution as a common yardstick. Imperfect, but at least it applies one model to every channel — Multi-Touch Attribution
  • Incrementality Testing and Geo Holdout Tests for anything material. The only method that answers what a channel caused
  • Marketing Mix Modelling at portfolio level, which needs no user-level data at all and is therefore immune to all of this
  • Blended efficiency — total spend against total revenue — as a sanity check. Crude, unattributable, and it can’t be inflated by anyone’s model

The reconciliation conversation

You will be asked why the platform says 620 and analytics says 340. The answer isn’t a fix, it’s an explanation, and it’s worth having ready:

  1. Different attribution. They credit any contact in their window; your analytics credits by its own model
  2. Different windows. Theirs is probably longer, and may include view-through
  3. Modelling. Some of their figure is estimated
  4. Tracking loss. Some of yours is genuinely missing — consent, blocking, delivery — Ad Blockers and Tracking Loss

Only point 4 is a problem you can fix. Points 1 to 3 are expected and permanent.

Track the ratio rather than the difference. If Google’s figure is consistently 1.8× yours, that’s a stable relationship you can reason with. When it moves to 2.4×, something changed — their model, your tracking, or the traffic — and that’s the signal worth watching, not the gap itself.

Graph View

Backlinks