Consent Banner A/B Testing: What to Measure Besides the Accept Rate

The new banner wins the A/B test: accept rate up nine points. Revenue from consented users falls. The test measured the wrong thing, and the banner that looked better was the one that collected consent from people who never read it, then lost them at the next step because the experience after consent got worse.

Why the accept rate lies

The accept rate measures how many visitors click accept, and it rewards the banner that is hardest to refuse: the one with a giant accept button, a greyed-out reject link, and a design that makes declining feel like work. Those banners win the test and lose the argument, because consent obtained through friction is consent a regulator can question, and because the users who accept under pressure behave differently from the users who accept because the choice was clear. They bounce more, they convert less, and they revoke more often, which is the number the accept rate never shows.

The accept rate also hides the denominator problem. A banner that shows to everyone and gets 60 percent accept looks better than a banner that shows only to EU visitors and gets 50 percent, even though the second one is measuring a different population. Geo-targeting changes the denominator, and any test that does not control for who saw the banner is comparing two different audiences and calling it a result.

The five numbers that matter

First, the reject rate, measured beside the accept rate, not instead of it. A banner with a 60 percent accept rate and a 35 percent reject rate is a different banner from one with 60 percent accept and 5 percent reject, because the second one has 35 percent of visitors doing neither, which usually means they closed the tab or the banner never rendered. Second, consent rate by category: analytics, marketing, and preferences separately, because a banner that wins accept on analytics and loses it on marketing has not won anything for the ad team.

Third, time to decision. A banner that takes eleven seconds to decide is a banner people are reading, and a banner that takes under a second is a banner people are dismissing. Both can be fine, but they mean opposite things, and the accept rate alone cannot tell them apart. Fourth, the revocation rate: how many users come back and change their choice, which is the best proxy for whether the consent was informed. Fifth, consented revenue: the revenue from users who consented, tracked as its own segment, because that is the number the banner exists to protect. The banner that maximizes accept rate and minimizes consented revenue is a failure wearing a win. The reject-button design problem is Reject All Button Design: Deceptive Patterns, Named.

The reject rate is the number most teams never look at, and it is the one that tells you whether your banner is honest. A banner that reports a 90 percent accept rate with a prominent Accept button and a reject option hidden two clicks deep is not measuring consent. It is measuring how many people gave up looking for the way out. Track the reject rate alongside the accept rate, and the share of visitors who made a real choice, accept or reject, in a reasonable time. A banner people dismiss in under a second is a banner they did not read, and a banner they did not read will not survive a regulator looking at it.

How to run the test without fooling yourself

Test one thing at a time: the button copy, the layout, or the timing, never all three, because a test that changes everything attributes the result to nothing. Run it for at least two full weeks, because consent behavior differs on weekdays and weekends, and a test that runs Monday to Friday measures office workers. Segment by region, because an EU visitor and a US visitor face different defaults and different expectations, and a single blended number hides both.

And keep the control honest. The control banner is the one you have now, not a deliberately bad one, because a test against a straw man proves nothing. The question is not whether the new banner beats a broken one. It is whether it beats the one that is already working, on the numbers that matter, and the answer has to survive contact with revenue. For the cookie decision that sits underneath all of this, see Third-Party Cookies Are Not Going Away. Consent Still Matters..

Frequently asked questions

Why is the accept rate a bad metric for consent banners?

It rewards banners that are hard to refuse, hides the reject rate and the people who do neither, and says nothing about whether the consent was informed or whether consented users convert.

What should I measure instead of the accept rate?

Reject rate, consent rate by category, time to decision, revocation rate, and consented revenue as its own segment. Together they show whether the banner collects real consent or just clicks.

How long should a consent banner A/B test run?

At least two full weeks, because consent behavior differs on weekdays and weekends. Segment by region too, since EU and US visitors face different defaults.

Does a high accept rate ever hurt revenue?

Yes. Banners that maximize accept through friction often collect consent from people who then bounce, convert less, or revoke. Consented revenue can fall while the accept rate rises.