Tags: statistics analytics concept
Ratio Metrics
Date: 2026-08-16
Any metric that’s one thing divided by another. Two traps: the average of ratios isn’t the ratio of the sums, and when the denominator varies per unit the standard error is wrong — which means the significance your tool reports is too generous.
What it is
A ratio metric is a quotient: conversion rate, revenue per visitor, items per order, click-through rate.
The complication is what the denominator is:
| Type | Example | Denominator |
|---|---|---|
| Fixed per unit | Conversion rate (users) | 1 per user — behaves like a proportion |
| Varying per unit | Items per order, revenue per session | Varies by unit — this is where it breaks |
Trap 1: averaging ratios
orders items items/order
mobile 800 1,200 1.50
desktop 200 800 4.00
average of ratios (1.50 + 4.00) / 2 = 2.75 ✗
ratio of sums 2,000 / 1,000 = 2.00 ✓
The correct answer weights by denominator. The naive average treats a segment of 200 orders as equal to one of 800.
In plain terms: you can’t average percentages unless the things underneath them are the same size. The site-wide rate is the total over the total, never the mean of the segment rates.
This is also the mechanism behind Simpson’s Paradox — when the weights change between periods, the aggregate moves independently of any segment.
Trap 2: the variance is wrong
The statistical one, and it’s the reason this note exists.
Standard tests assume each unit contributes one independent observation. With a varying denominator that breaks:
revenue per SESSION, randomised by USER
user A 1 session, £40
user B 6 sessions, £240
user B contributes six observations to the denominator
and they are not independent — same person
The result: the naive standard error is too small, so intervals are too narrow and p-values too small. Your tool reports significance you haven’t earned, with no warning.
The clean fix is to match the analysis unit to the randomisation unit — analyse per user, not per session. Where you genuinely need a session-scoped ratio, the correct standard error requires the delta method or Bootstrapping, and most platforms do neither by default.
Worth checking what yours does. See Randomisation Unit and Sessionisation.
Which ratios are safe
SAFE — denominator is 1 per randomised unit
conversion rate (users who converted ÷ users)
bounce rate per user
RISKY — denominator varies per unit
revenue per session (sessions vary per user)
items per order (orders vary per user)
click-through per pageview
The test: is the denominator the thing you randomised on? If yes, it’s a proportion and the standard methods apply. If no, the variance calculation needs care.
Practical
- Prefer user-scoped ratios in experiments. It removes the problem rather than correcting for it
- Compute site-wide as total ÷ total, never as an average of segment rates
- Report the numerator and denominator alongside the ratio. A 12% rate on 40 sessions and on 40,000 are different findings, and only the counts distinguish them — Sampling Error
- Watch the denominator move. A ratio can change because the bottom moved rather than the top — the Sessionisation worked example, where a more interactive variant reduces session count and raises session conversion rate with no additional orders
- Bootstrap when in doubt. It handles ratio variance correctly without needing the delta method’s algebra
Where it appears
- Metric Design — the denominator is one of the seven fields, and this is why
- Metric Sensitivity — ratio variance affects how much traffic you need
- Revenue Metrics — revenue per visitor is the ratio metric that matters most and behaves worst
- Percentiles in Performance — the related “you can’t average percentiles” problem