Split test sample size and duration calculator for landing pages

Your new lander converts at 12% and the old one at 9% after 300 clicks. That looks like a win, and it may be noise: at that traffic a gap of three points comes and goes from one day to the next. The fix is to decide how many visitors each page needs before you launch, then wait for them. Unbounce gives the same advice in its landing page testing guide: estimate the traffic and the run time before the test starts, not halfway through.

Once you have the number, you still need a builder that can split traffic without breaking your ad tracking. The split-testing column in the comparison table shows which builders do it and on which plan.

The method: a two-proportion z-test with a fixed sample

A landing page test compares two conversion rates. Each visitor either converts or does not, so each page's rate is a binomial proportion, and Evan Miller's How Not To Run an A/B Test gives its variance as the rate times one minus the rate. Optimizely's notes on sample size calculations describe the same setup for a fixed-horizon test: a Wald test, also called a z-test, with the sample size chosen before the first visitor arrives.

The calculator takes your current rate as p0 and the rate you hope the new page reaches as p1 = p0 × (1 + lift). It then works out the visitors each page needs:

n per page = (z_conf + z_power)² × [p0(1−p0) + p1(1−p1)] / (p1 − p0)²
total      = n per page × number of pages
days       = total / visitors per day, rounded up

At 95% confidence z_conf is 1.96, and at 80% power z_power is 0.84 (the script carries more decimals, 1.95996 and 0.84162). The 90% and 99% confidence options use 1.64485 and 2.57583, and 90% power uses 1.28155. Unbounce's A/B test duration calculator explains the two settings in plain terms: confidence is how sure you can be that a result is real, and power is the chance the test catches a lift that does exist.

This is the classic fixed-horizon calculation. It is the right tool when the goal is a yes-or-no conversion such as a lead or a sale. It does not cover sequential testing that lets you peek, Bayesian testing, or revenue per visitor, where the spread of order values matters as much as the rate. With three or more pages it sizes each comparison against the control and makes no correction for running several at once, so treat a narrow win among many pages with extra care.

Worked example: a 2.5% lander on 400 visitors a day

Say an affiliate lander converts 2.5% of its paid clicks to the offer page, and anything under a 10% relative lift (2.5% to 2.75%) is not worth switching for. Two pages, 95% confidence, 80% power, 400 visitors a day to the test URL.

• The gap to detect is 0.25 percentage points, and the combined variance term is 0.0511.
• That works out to 64,196 visitors for the control and 64,196 for the challenger, 128,392 in total.
• At 400 a day that is 321 days, close to a year.

Now aim for a 20% lift instead (2.5% to 3%). The gap doubles and the sample falls to roughly a quarter, because the gap is squared in the denominator: 16,789 visitors a page, 33,578 in total, 84 days at the same traffic. A higher baseline helps too: the same 10% lift on a 10% page needs 14,749 visitors a page.

The lesson for low-converting paid traffic is plain. Small tweaks to a 2% page are close to untestable on a few hundred clicks a day, so test changes big enough to move the rate by a fifth or more.

How long to run a landing page split test

The arithmetic is total visitors divided by daily visitors, rounded up to a whole day. Unbounce adds a calendar floor on top: run for at least one to two weeks even when the sample fills sooner, so every day of the week is in the data. Practitioners on r/PPC land in the same place and suggest two weeks before reading a landing page result. Our rule of thumb: use whichever is longer, the calculated days or 14, and end on the same weekday you started.

Google Ads experiments need extra room. The Google Ads experiments FAQ says the first seven days of data are thrown away to allow for ramp-up, so a test that needs 21 days of counted traffic takes 28 on the calendar. Tick the box in the calculator and it adds that week for you.

Whether you split one URL server-side or send traffic to two URLs does not change the sample size. It changes what the test does to your ad campaign, a separate choice covered in our landing page builder rankings.

When the answer is longer than your budget

A result of 200 days is information, and it tells you this test should not run as designed. You have three honest options.

• Test a bigger change. A new headline and offer angle can move a rate by 20% or more. A button colour rarely will, and the calculator shows what detecting a small lift costs.
• Buy or pool more traffic. Send every campaign for the offer to the test URL, or accept a run of several weeks and plan the budget for it.
• Measure a step with a higher rate. Clicks through to the offer convert far more often than final sales, so the same lift is detected with fewer visitors. It is still a yes-or-no event, and the method holds, but confirm later that the sales follow.

Cutting the run short to get an answer is the one option to rule out. Evan Miller's article shows why: if you check a running test and stop the moment it looks significant, the significance figure no longer means what it says, and fixing the sample size in advance removes that problem.

If visitor caps worry you on a long test, the builder cost calculator shows what each builder bills at your monthly traffic once caps and overage are counted, and the individual builder reviews show where split testing switches on.

Sources and further reading

• Optimizely: sample size calculations for experiments, the Wald (z-test) setup for fixed-horizon tests.
• Evan Miller: How Not To Run an A/B Test, on binomial variance and why the sample is fixed in advance.
• Unbounce: A/B test duration calculator, definitions of confidence and power and the one-to-two-week floor.
• Google Ads Help: experiment results, where a variation counts as significant at 95%.
• Google Ads Help: experiments FAQ, on the seven discarded days.

Keep reading: all free calculators for paid-traffic landers, the full builder ranking, and how we research builders.