Growth Notes

Experiment Assignment Tracking

Tags: experimentation analytics concept

Experiment Assignment Tracking

Date: 2026-08-16

Recording who saw which variant, and when they actually saw it. Without it there is no analysis at all — and the difference between logging assignment and logging exposure silently halves or doubles your measured effect.

What it is

Experiment assignment tracking is the record linking each unit to the variant it received, joinable to that unit’s subsequent behaviour.

It’s the join key for the entire analysis. Everything else — the effect, the interval, the SRM check — is computed from this table joined to the events table.

ASSIGNMENT EVENTS                          joined to OUTCOME EVENTS

user_id  experiment      variant  ts       user_id  event      ts
u_8421   checkout-cta-08  variant  09:02   u_8421   purchase   09:14
u_1150   checkout-cta-08  control  09:03   u_1150   —
u_9930   checkout-cta-08  variant  09:03   u_9930   purchase   09:31

Assignment is not exposure

The distinction that matters most, and the one most implementations get wrong.

  • Assignment — the unit was bucketed into a variant. May have happened on the homepage
  • Exposure — the unit actually reached the thing being tested and could have been affected by it

If you test a checkout change but log assignment for every visitor, you’re including everyone who never reached checkout. They convert identically in both arms by construction, because they never saw the difference.

Worked, on a checkout test with a genuine 10% relative lift among people who reach checkout:

                        assigned   reach checkout   true lift among those
all visitors             100,000            20,000          +10%

MEASURED ON ASSIGNMENT           dilution factor 20,000/100,000 = 0.2
  observed lift  ≈  10% × 0.2  =  +2%      ← needs ~25× the sample to detect

MEASURED ON EXPOSURE
  observed lift  =  +10%                    ← detectable

In plain terms: including people who couldn’t possibly have been affected waters the effect down towards nothing. The test then reports “no difference” for a change that worked.

The rule: fire the exposure event at the moment the variant could first influence behaviour, not when the SDK decides the bucket. On the checkout page render, not on session start.

The opposite error

Firing exposure after the variant has already had its effect. If the exposure event fires on click of a button the variant made more prominent, then the variant’s users are exposed more often, and you’ve made exposure an outcome. The arms are no longer comparable and it produces spectacular fake wins.

Test for it: could the treatment change who gets logged as exposed? If yes, move the exposure point earlier.

Requirements

  • Fire once per unit per experiment, deduplicated. Repeated exposure events distort counts and any per-exposure metric
  • Include the experiment key and variant, so concurrent tests are separable
  • Same mechanism in both arms. If the variant path loads extra JavaScript that fires exposure, and control’s doesn’t, you have an asymmetry that presents as Sample Ratio Mismatch
  • Survive the page. Exposure firing on unload or after a redirect loses users unevenly — see Event Batching and Delivery
  • Server-side where possible. Client-side exposure events are lost to ad blockers at a rate that differs by variant if the variants differ in what they load — Ad Blockers and Tracking Loss
  • Joinable to conversions. A backend purchase event carrying only a user ID cannot join to an exposure event carrying only a cookie ID. This is Identity Stitching and it’s the most common reason an analysis silently drops conversions

Where it breaks

  • Assignment logged, exposure never implemented — dilution, as above. The commonest and most expensive
  • Exposure logged only on the variant — control has no denominator, and the tool silently substitutes something wrong
  • Bots and internal traffic included — both arms polluted, effects diluted, and if filtering happens after assignment it can create SRM — Bot and Internal Traffic
  • Assignment fired on every page view — inflates counts and breaks any per-user analysis
  • Exposure fired before the variant renders — users who bounced before seeing anything are counted as exposed, which is dilution again in miniature

The check that catches most of this: exposed counts per arm should be close to equal, and both should be materially smaller than assigned counts. If exposed equals assigned, exposure isn’t really being tracked.

Graph View

Backlinks