Segmentation (analysis)
Date: 2026-08-16
Cutting the population to find the group the average was hiding. It’s the highest-yield analytical move there is, and it’s the same operation that manufactures false findings — the difference is entirely whether you decided the cut before or after seeing the data.
What it is
Segmentation is splitting a population into subgroups and comparing a metric across them.
The reason it’s so productive: an average describes nobody. A 3% site conversion rate might be 4.8% desktop and 1.9% mobile, and every decision made on the 3% is made about a customer who doesn’t exist.
The segments that pay
For retail, roughly in order of how often they reveal something:
- Device — mobile and desktop are effectively different sites, with different rates, baskets and behaviour
- New versus returning — different intent entirely, and the split that detects Novelty and Primacy Effects
- Channel — arriving from branded search versus paid social are different populations before they land — Channel Taxonomy
- Template — homepage, category, product, search. Site-wide numbers hide which page is failing
- Geography — market, and for performance work, distance to origin
- Basket value band — high-value customers behave differently and are often a minority of orders and a majority of revenue
- Logged in versus anonymous — with the caveat that these are self-selected — Anonymous and Identified Users
The trap
Post-hoc segmentation on a test result is a different activity, and it generates false findings reliably — fifteen slices gives you better than a 50% chance of a significant one by chance. That’s covered properly in Segmentation (test results).
In observational analysis the same statistical problem exists but the stakes differ: you’re generating hypotheses rather than concluding. The discipline that keeps it honest:
A segment finding is a hypothesis until it’s replicated or tested. “Mobile users on paid social convert 40% worse” is a lead worth pursuing, not a fact worth acting on. See The Multiple Comparisons Problem.
Simpson’s paradox
The specific failure worth watching for: a metric can move in one direction in every segment and the opposite direction overall, when the mix between segments changes.
mobile desktop total
last month 1.8% 4.9% 3.0% (40% mobile)
this month 1.9% 5.0% 2.7% (60% mobile)
↑ up ↑ up ↓ down
Both segments improved. The total fell because mobile — the weaker converter — grew as a share of traffic.
Always check the mix before interpreting an aggregate change, and report segment sizes alongside segment rates. See Simpson’s Paradox.
Practical
- Segment before concluding anything, as a habit. Any site-wide number is an average over populations that differ
- Report counts alongside rates. A 12% conversion rate on 80 sessions is noise, and rates alone hide it — Sampling Error
- Watch for correlated segments. Mobile, paid social and new visitor are largely the same people. “It’s worse across three segments” is often one finding counted three times
- Prefer segments you can act on. “Users on Android 9” is a real segment and rarely a decision. Device, channel and template map to things you can change
- Cross two dimensions at most. Three-way splits produce cells too small to read and multiply the comparison count
Where it stops
Segmentation finds where. It never finds why — for that you need Session Replay, Form Analytics or Usability Testing.
And it can’t establish that a segment’s difference is caused by the segment. Mobile users converting worse may be about the mobile experience, or about mobile traffic being earlier in the consideration cycle. Distinguishing those requires an intervention, not a finer cut — Controlled Experiments.