Tags: statistics concept

Variance and Standard Deviation

Date: 2026-08-16

---

How spread out the data is. It’s the number that decides how much traffic a test needs — so for anyone running experiments, variance matters more than the average does.

---

What it is

Variance is the mean squared distance from the mean. Standard deviation is its square root, which puts it back in the units of the original data.

${\sigma }^2=\frac{\sum (x_i-\bar{x}{)}^2}{n}\sigma =\sqrt{{\sigma }^2}$

Worked, on five order values:

values          £20   £30   £40   £50   £60
mean                        £40

deviations      −20   −10     0   +10   +20
squared         400   100     0   100   400     sum = 1,000

variance        1,000 ÷ 5              = 200
std deviation   √200                   = £14.14

Use the standard deviation to describe, the variance to calculate. £14.14 is interpretable; 200 “squared pounds” isn’t. But variance is what appears in every sample size formula, because variances add and standard deviations don’t.

Why squaring

Deviations sum to zero by construction, so you need to remove the signs. Squaring rather than taking absolute values makes the maths tractable — variances of independent things add, which is what makes the whole apparatus work.

The cost: squaring weights large deviations disproportionately. One order at £5,000 contributes far more to the variance than fifty orders at £60. That’s the mechanism behind heavy tails wrecking averages — Skewed and Heavy-Tailed Distributions.

n or n−1

population variance    ÷ n        you have every member
sample variance        ÷ (n − 1)  you're estimating from a sample

Dividing by n−1 corrects a downward bias — a sample’s spread around its own mean is systematically smaller than its spread around the true mean. Every tool defaults to n−1 and you’ll rarely think about it, but it’s why two tools can differ in the last decimal.

Why it decides your traffic bill

The practical payoff. Sample size scales directly with variance:

$n\propto {\sigma }^2$

So halving the standard deviation quarters the sample needed. From Metric Sensitivity:

Metric	σ	n per arm
Conversion rate (3%)	0.171	53,000
Revenue per visitor	£13.44	126,000

Same site, same minimum detectable effect (MDE). The entire difference is spread.

In plain terms: a test doesn’t need lots of people because the effect is small. It needs lots of people because customers differ from each other, and you have to see through that noise to find the effect. Reduce the noise and you need fewer people.

That’s why variance reduction is the only lever that buys power for free — Variance Reduction, Winsorisation and Capping.

The coefficient of variation

Standard deviation relative to the mean:

$CV=\frac{\sigma }{\mu }$

The comparable measure across metrics with different units. A £13 standard deviation means nothing until you know the mean is £1.50 — at which point CV = 9, and you know it’s a very noisy metric. See Metric Sensitivity.

For proportions it’s determined

A conversion rate has no free variance parameter — it’s fixed by the rate:

$\text{Var}=p(1-p)$

You cannot reduce the variance of a conversion rate by cleaning the data. It’s a property of the rate. That’s a real asymmetry with continuous metrics, where capping and covariates genuinely help — Binomial and Bernoulli Distributions.

Where it appears

• Standard Error — the standard deviation of a statistic, not of the data
• Sample Size Calculation — variance is one of the four inputs
• Confidence Intervals — the width is a multiple of the standard error
• Percentiles in Performance — for skewed data, spread is better read from percentiles than from σ