Tags: statistics commerce concept

Skewed and Heavy-Tailed Distributions

Date: 2026-08-16

---

Revenue, session duration, page load, order value — the metrics that matter commercially are the ones where the average lies. A handful of extreme values carry most of the total, and every standard method assumes they don’t.

---

What it is

A skewed distribution is asymmetric — one tail is longer than the other. Heavy-tailed means extreme values occur far more often than a normal distribution would predict.

Almost everything commercially interesting is right-skewed:

order value        ▌▖
                   ▌▝▖▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁
                   most orders small, a few enormous

page load          ▌▖
                   ▌▝▖▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁
                   most fast, a few catastrophic

session duration   ▌
                   ▌▖▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁
                   most seconds, a few hours

The cause is usually multiplicative: values are bounded below at zero and unbounded above, and they compound rather than add.

What it breaks

The mean stops describing anything. With the tail dragging it, the mean can sit above the 90th percentile — most observations below the “average” — Mean Median and Mode.

Variance is enormous, so sample size requirements explode. Revenue per visitor needs 2.4× the traffic of conversion rate for the same relative lift, entirely because of spread — Metric Sensitivity.

The CLT converges slowly. Sample means do go normal eventually, but “eventually” means thousands or tens of thousands of observations for heavy tails. Below that, normal-approximation confidence intervals are too narrow — they understate uncertainty, which is the dangerous direction.

Results are unstable. One £5,000 order arriving in one arm rather than the other can flip a revenue test. That’s not a signal; it’s a single customer.

In plain terms: with heavy-tailed data, your result depends heavily on which few large values happened to land in which arm. Rerun the same test and you’d get a materially different answer.

Detecting it

Three checks, all quick:

• Mean versus median. A large gap means skew. Mean above p75 means severe skew — Percentiles and Quantiles
• Top 1% share. What proportion of total revenue comes from the top 1% of orders? Above 10–15% and the tail is doing real work
• Plot it. A histogram takes thirty seconds and shows the shape, the tail, and any second population

Handling it

In rough order of preference:

Change the metric. Conversion rate has no tail. Frequently the honest answer is that revenue per visitor was the wrong primary metric for the traffic available — Overall Evaluation Criterion.

Winsorisation and Capping. Cap at a pre-registered percentile so one order can’t decide a test. Set before you look, applied identically to both arms.

Bootstrapping. Build the confidence interval by resampling rather than by formula. No distributional assumption, and the honest route for revenue.

Report percentiles. Median and p75 rather than the mean, wherever you’re describing rather than summing.

Log transformation. Compresses the tail and makes the distribution near-normal — at the cost of a result in units nobody can interpret commercially, and a mean that doesn’t back-transform to the mean.

Separate the populations. A bimodal distribution — trade and consumer, say — isn’t one skewed distribution, it’s two distributions stacked. Segment them rather than modelling the mixture — Segmentation (analysis).

Where it appears

• Revenue and AOV — Revenue Metrics, Average Order Value
• Performance — every timing metric, which is why the field standardised on p75 — Percentiles in Performance
• Session duration and time on page — Engagement Metrics
• Customer value — a small share of customers generate most of it, which is why Customer Lifetime Value is so sensitive to how the tail is treated