Tags: statistics concept
Communicating Uncertainty
Date: 2026-08-16
Presenting a range to people who want a number. Done badly it reads as “you don’t know” and gets discarded; done well it’s the thing that makes you trusted when a result later moves.
What it is
Communicating uncertainty is conveying the precision of a result alongside the result, so decisions are made on what the evidence actually supports.
The failure isn’t statistical — it’s that a range feels like an admission of weakness, so people round it away and the decision gets made on a point estimate the data never supported.
Translate before you report
The single most effective move. Nobody acts on percentage points; everyone acts on money.
STATISTICAL COMMERCIAL
+3.0% relative, between £2,000 and £43,000
CI +0.3% to +5.7% a year, most likely around
£22,000
The second is the same information and it’s decision-ready. Do the multiplication yourself — annual visitors × baseline × order value × lift, then margin if the decision needs profit — rather than leaving it to the reader, who will otherwise anchor on the point estimate and forget the range entirely.
500,000 visitors × 3% × £50 = £750,000 a year through this page
× 0.3% (interval, lower) = £2,250 ≈ £2,000
× 3.0% (point estimate) = £22,500 ≈ £22,000
× 5.7% (interval, upper) = £42,750 ≈ £43,000
In plain terms: the test can’t say exactly what the change is worth. It can say it’s very probably worth something, probably not more than about £43,000 a year, and possibly as little as £2,000..
The four-part statement
A defensible result has four components, in this order:
“The variant increased conversion by 3.0% relative (estimate), with a 95% interval of +0.3% to +5.7% (uncertainty). At current traffic that’s £2,000 to £43,000 a year (commercial translation). Given the £15,000 build cost, I’d re-run with more traffic before committing (recommendation).”
The recommendation is what stops the range reading as indecision. A range plus a clear course of action is confident; a range alone is not.
Framing that works
- Lead with the decision, not the statistics. “I’d ship this” then the evidence, rather than making people wait for a verdict
- Use the interval’s bounds as scenarios. “Worst case we make £2,000, best case £43,000” is concrete where “the confidence interval is +0.3% to +5.7%” is abstract
- Say what would change your mind. “If it holds above +2% after another month, I’d commit” gives people a reason to wait rather than a refusal to decide
- Give the null result content. “We can rule out anything above 1% in either direction” is a finding. “Not significant” is not — Inconclusive Results
Language to avoid
| Don’t say | Because |
|---|---|
| ”95% confident the true lift is 3%“ | That isn’t what a confidence interval means — Confidence Intervals |
| ”The test proved it works” | Nothing proves anything — Hypothesis Testing |
| ”Not significant, so no effect” | Absence of evidence isn’t evidence of absence — Statistical Power |
| ”The data shows…” for a post-hoc segment | It shows a hypothesis — Segmentation (test results) |
| “Statistically significant” as a headline | Heard as “important” rather than “detectable” — Practical vs Statistical Significance |
Visually
- Show the interval, as error bars or a shaded band. A bar chart of two point estimates implies a precision that doesn’t exist
- Never truncate the y-axis on a comparison. It manufactures a visual difference the data doesn’t support
- Show counts alongside rates, so small samples are visibly small — Sampling Error
- Annotate the timeline so step changes have context rather than inviting invention — Annotation and Change Logs
The long game
The reason to do this properly isn’t rigour for its own sake. It’s that results move. A shipped winner underdelivers — expected, via Winner’s Curse — and what happens next depends entirely on what you said at the time.
If you reported “+3%, worth £22,000”, you were wrong and your next number is discounted. If you reported “somewhere between £2,000 and £43,000, and I’d verify before committing”, you were right and the process worked.
Stating uncertainty is what makes you credible when the estimate moves, which it will. That’s the argument that actually persuades people to do it — not the statistics.