Tags: statistics concept

The Normal Distribution

Date: 2026-08-16

---

The shape every method defaults to. It earns that position not because data is usually bell-shaped — it usually isn’t — but because averages are, which is a different and much more useful fact.

---

What it is

The normal distribution is a symmetric, single-peaked distribution defined entirely by two numbers: its mean (μ) and its standard deviation (σ).

Its useful property is that those two numbers fix everything about it:

              68%
        ┌──────────────┐
        │   ╱▔▔▔▔▔╲    │
   ╱────┴──╱       ╲───┴────╲
 ──┴───────┴───────┴───────┴──
  −2σ     −1σ   μ   +1σ    +2σ

  ±1σ  →  68% of values
  ±2σ  →  95%
  ±3σ  →  99.7%

That’s where 1.96 comes from — the multiplier in every 95% confidence interval is the number of standard deviations containing 95% of a normal distribution — Confidence Intervals.

Why it dominates the methods

Not because your data is normal. Because sample means are, almost regardless of what the underlying data looks like — that’s The Central Limit Theorem, and it’s the single most consequential result in applied statistics.

raw conversions      0, 1, 0, 0, 1, 0 ...     Bernoulli, not remotely normal
sample conversion
rate across many
hypothetical samples  3.1%, 2.9%, 3.0% ...    approximately normal

Hypothesis tests operate on the second, not the first. So “is my data normal?” is usually the wrong question — the right one is “is my sample statistic approximately normal at this sample size?”

In plain terms: individual customers are nothing like a bell curve. The average of thousands of them is, and the average is what you’re testing.

When the approximation holds

Situation	Normal approximation
Conversion rate, thousands of users	Fine
Conversion rate, very low base rate, small n	Poor — too few successes
Revenue per visitor, thousands of users	Marginal — heavy tail converges slowly
Revenue per visitor, hundreds	Poor
Order value of individual orders	Never — it’s log-normal, not normal

Rough guide for proportions: you want at least 10 expected successes and 10 expected failures in each arm. At a 0.5% conversion rate that needs 2,000 users per arm before the approximation is even reasonable.

For revenue, the heavy tail means convergence is slow and intervals from the normal approximation are too narrow — they understate uncertainty, which is the dangerous direction. That’s the argument for Bootstrapping — see Skewed and Heavy-Tailed Distributions.

What it isn’t

Three misreadings worth clearing:

• “Normal” doesn’t mean typical or correct. It’s a name, not a judgement. Skewed distributions are entirely normal in the ordinary sense of the word
• Real data is never exactly normal. It’s an idealisation; the question is always whether the approximation is close enough for the sample size
• Normality of the raw data is rarely required. What matters is the sampling distribution of the statistic

Checking it

Rarely necessary for large-sample proportion tests, and worth doing before trusting a normal-based interval on anything continuous:

• Plot a histogram. Thirty seconds, and it catches skew and bimodality that no summary statistic will
• Compare mean and median. A large gap means skew — Mean Median and Mode
• Look at the tail. If the top 1% of values is a large share of the total, the tail is heavy and the mean is fragile

If it fails, the options are Bootstrapping, a non-parametric test, a transformation, or Winsorisation and Capping — in roughly that order of preference.