Tags: analytics commerce concept

Cohort Analysis

Date: 2026-08-16

---

Group people by when they started, then track each group separately. It’s the fix for mix changes masquerading as behaviour changes — and the mix change it can’t fix is the one in your own measurement.

---

What it is

A cohort is a set of users sharing a start point — usually the month they first purchased. Cohort analysis tracks each cohort’s behaviour separately over subsequent periods.

              month 0   1      2      3      4
Jan cohort      100%   32%    24%    19%    17%
Feb cohort      100%   35%    26%    21%
Mar cohort      100%   34%    25%
Apr cohort      100%   29%

Read down a column to compare cohorts at the same age. Read across a row to see one cohort mature.

Why it beats an aggregate

An aggregate metric mixes cohorts of different ages, so it moves when the mix moves rather than when behaviour does.

overall repeat rate falls from 28% to 24%

  aggregate reading:   "retention is getting worse"

  cohort reading:      every cohort's month-1 rate is flat or
                       rising. A large new-customer intake
                       diluted the average with month-0 users

A successful acquisition month makes aggregate retention look worse. That’s Simpson’s Paradox in its most common commercial form, and cohorts are the standard defence.

Choosing the cohort definition

• Acquisition date — first purchase or first visit. The default, and right for retention and lifetime value
• Behavioural — everyone who used a feature, or bought a category. Right for “does this behaviour predict retention”, but selection is on behaviour, so the comparison isn’t causal
• Grain — monthly for most retail; weekly for high-frequency; quarterly for long consideration cycles. Too fine and cohorts are too small to read — Sampling Error

The measurement artefact that ruins it

The one specific to analytics rather than to the method.

Identity improves over the life of an account. Logins accumulate, devices get stitched, so a cohort measured six months in has better identity resolution than the same cohort measured at month one.

Jan cohort measured in Feb    identity partial   → looks like more users, less activity each
Jan cohort measured in Aug    identity better    → fewer users, more activity each

That reads exactly like improving retention and engagement. It’s Identity Stitching getting better, and it biases every cohort chart in the same direction.

Guard against it by fixing the identity resolution date — compute all cohorts using the stitching state as of one date — or at minimum by knowing the bias exists before interpreting an upward trend.

Related traps:

• Only identified users have cohorts. Anonymous traffic can’t be tracked over months, so cohort analysis silently describes your loyalists — Anonymous and Identified Users
• The most recent cohorts are incomplete. April’s month-3 cell doesn’t exist yet. Charts that plot it as zero, or averages that include it, are wrong
• Retention windows and data retention interact. Your data retention setting is a hard ceiling on cohort age — a 14-month window means no cohort older than 14 months exists to analyse [CHECK: your tool’s actual retention setting, which is often shorter than assumed] — Data Retention

What to look for

• Down the columns — is month 1 improving cohort on cohort? That’s the clearest signal a change worked
• The shape of the curve, not the level — whether it flattens is the question that matters — Retention Curves
• Revenue as well as count. A cohort with fewer returning customers spending more is a different business than the reverse — Cohort Revenue
• Cohort size alongside rates. A brilliant retention rate on 40 customers is noise

Where it’s strongest

Judging whether an acquisition or lifecycle change worked. Aggregate metrics can’t separate “we acquired more people” from “the people we acquire are better”, and the distinction is the whole question for a growth programme. See Retention and Churn and Customer Lifetime Value.