Tags: experimentation ux concept
Painted Door Tests
Date: 2026-08-17
Advertising something that doesn’t exist to measure whether anyone wants it. The entry point is real, the destination isn’t — a link, a button, a plan tier that leads to “coming soon”. It’s the cheapest demand signal available, and it works by deceiving your customers, which is a cost rather than a technicality.
A painted door test shows users an entry point to a feature or offer that doesn’t exist yet, and measures how many try to use it.
The shape
REAL BUTTON NO PRODUCT BEHIND IT
┌──────────────────────────┐
│ Subscribe & save 10% │ ← click tracked as the signal
└──────────────────────────┘
↓
┌──────────────────────────────────────────────┐
│ Subscriptions are coming soon. │ ← the honest landing
│ Want to know when? [email] │ ← the consolation, and
│ Sorry for the dead end — here's 10% off │ a second, stronger signal
│ your next order for the trouble. │
└──────────────────────────────────────────────┘
Variants of the same idea: a fake door (as above), a concierge test (the feature is real to the customer and performed manually behind the scenes), and a smoke test landing page (an advert and a page for a product that isn’t built).
The concierge version avoids the deception entirely and should be the first thing considered — the customer genuinely gets what they asked for, just delivered by a person rather than a system. It doesn’t scale past a few dozen, which is usually enough to answer the question.
What the number does and doesn’t mean
The click rate measures interest at zero commitment, which overstates real demand by a wide and unknown margin.
subscribe button impressions 84,000
clicks 2,940 3.5%
email captures on the dead end 610 20.7% of clickers, 0.73% of impressions
what you can claim:
✓ "3.5% of PDP visitors clicked a subscription entry point"
✓ "0.73% left an email address for it"
✓ "that's 4× the click rate on the gift-wrap entry point" ← comparison is the value
what you cannot claim:
✗ "3.5% of customers will subscribe"
✗ any revenue projection from the click rate
Interpret it comparatively, never absolutely. Painted door numbers are meaningful against another painted door — this feature versus that one, this position versus that one — and meaningless as a forecast. The gap between clicking and buying is enormous and varies by category.
The friction ladder is the useful refinement. Each additional step someone completes is a stronger signal, and the drop-off between steps is more informative than any single rate:
sees it → clicks → gives email → picks a plan → enters card details
weak ─────────────────────────────────────────────────────→ strong
A test that stops at “clicked” is cheap and weak. One that gets to “entered card details, then told it’s not ready” is a much stronger signal and a much larger breach of trust — the strength and the ethical cost rise together, which is not a coincidence.
The ethics, stated properly
This is a design that works by misleading people, so the question isn’t whether that’s true but whether it’s proportionate.
What makes one defensible:
- The dead end is immediate and honest. “This isn’t built yet, we’re gauging interest” — not a vague error, not a spinner, never a fake 404
- Something is offered in return. Early access, a discount, or at minimum a clear “we’ll tell you when”. The user gave you information; give something back
- It’s small, capped and short. A 5% exposure for two weeks, not a permanent fixture
- Nothing is taken. No payment details captured for a product that doesn’t exist. This is where “cheeky test” becomes something a regulator would take an interest in
- Not on a critical path. Never interrupt a checkout with a door that goes nowhere
- It’s removed afterwards, and the people who signed up are actually contacted if it ships
What makes one indefensible: repeat exposure to the same users, wasting substantial customer time, tests on vulnerable users or in high-stakes contexts, and anything that induces a purchase decision on a false premise. The reputational cost is real and asymmetric — the screenshot of your fake button lives longer than the insight.
In the UK, presenting something as available when it isn’t can engage consumer protection rules on misleading actions and omissions, and advertising codes if it appears in paid media. The distinction that matters is between gauging interest in a clearly-flagged upcoming feature and representing a product as purchasable. [CHECK: which specific regime applies — consumer protection legislation was restructured recently and the enforcement body’s powers changed with it. Confirm current position before running anything that touches price, availability or payment.] — Testing and Compliance, Deceptive Design
Doing it well
- Run it as a proper B test, not as a one-off placement. Without a control you can’t tell whether the door cannibalised something else on the page
- Watch guardrails hard. The door takes attention from real conversion paths, and a “successful” painted door that costs 1% of orders has failed — Guardrail Metrics
- Vary position and framing across cells if you want to know whether it’s demand or prominence you measured
- Pair it with qualitative follow-up. The email addresses are a recruitment list for interviews, and “why did you click” is worth more than the click rate — User Interviews
- Pre-register what result triggers building it, before launch. Otherwise the number is read as whatever supports the decision already taken — Pre-Registration
Where it interacts
- Hypothesis Design — a painted door tests a demand hypothesis, and the mechanism has to be stated or the click rate is uninterpretable
- Ethics of Experimentation — this is the design where the ethical question is most acute, and it’s the reason that note exists
- Jobs To Be Done — the qualitative counterpart, and often the better first move
- Rollouts as Experiments — the honest alternative once something is real: build the smallest version and ramp it