A landing page A/B test is useful only when it produces a decision you can trust. This checklist shows how to form a testable hypothesis, estimate the traffic and conversions required, compare common page elements, record results in an experiment log, and decide when to keep, revise, or rerun a test.
Overview
Landing page A/B testing compares two or more page versions while keeping the main audience, traffic source, and conversion goal consistent. In a basic test, version A is the control and version B changes one defined element. Visitors are assigned to a version, their actions are recorded, and the results are evaluated against a preselected success metric.
The aim is not to find a universally “best” design. It is to learn which page experience performs better for a particular audience and campaign context. A headline that works for branded search traffic may not work for visitors arriving from a product launch email. Treat every result as evidence for a decision, not as a permanent rule.
Before launching, define four things:
- Primary conversion: the action the page is designed to generate, such as a form submission, trial signup, booking request, or purchase.
- Control and variation: the existing page and the deliberately changed version.
- Decision rule: the conditions required to ship the variation, continue testing, or keep the control.
- Test window: the planned period and the minimum amount of evidence required before reviewing the result.
Use a reliable analytics setup before you experiment. Your event names, UTM parameters, form-success events, and conversion definitions should be consistent across both versions. The landing page analytics setup guide can help you check those foundations.
How to estimate
Estimating an A/B test means translating current performance into a practical traffic and time plan. Start with the control conversion rate:
Control conversion rate = conversions ÷ eligible visitors × 100
For example, if 42 of 1,000 eligible visitors complete the primary action, the observed control rate is 4.2%. Next, state the smallest improvement that would justify changing the page. This is your minimum worthwhile effect. It might be a relative improvement, such as 15% more conversions, or an absolute improvement, such as an increase from 4.2% to 5%.
Do not choose the target solely because it makes the test finish quickly. A small page change may produce a small but valuable improvement, while a large claimed improvement may be unrealistic. Record the expected effect before viewing the variation’s results.
A simple planning estimate is:
Visitors needed per version = planned total visitors ÷ number of versions
To estimate duration:
Test duration in days = visitors needed per version ÷ average eligible visitors per day
These formulas are planning aids, not a substitute for statistical power calculations. Sample-size requirements depend on the baseline rate, the effect you want to detect, the number of variants, and the statistical method used by your testing platform. If the expected improvement is small or the baseline conversion rate is low, plan for substantially more observations and avoid declaring a winner from a short run.
Set the test duration around complete visitor cycles rather than stopping after a convenient result. Weekday and weekend audiences can behave differently, and campaigns may have delayed conversions. If the page has too little traffic for a controlled test, consider testing a larger change, improving measurement first, or using qualitative feedback and sequential experiments instead of overstating a weak result.
Inputs and assumptions
Use this pre-launch checklist to make the experiment reproducible:
- Audience: identify the source, campaign, geography, device mix, and eligibility rules. Exclude internal traffic and known test users where appropriate.
- Primary metric: choose one conversion that directly represents the page’s business goal. Keep supporting metrics, such as form starts or scroll depth, secondary.
- Baseline: calculate the control rate from a defined period and note whether the period includes a promotion, outage, seasonality, or unusual traffic source.
- Minimum worthwhile effect: state the smallest change that would affect the decision. Include the expected control rate and target variation rate.
- Allocation: decide how visitors will be split between versions. Avoid changing the allocation mid-test unless there is a documented technical or safety reason.
- Quality checks: verify that redirects, forms, tracking events, consent choices, mobile layouts, accessibility features, and page speed work in every version.
- Stopping rule: define the minimum sample, planned duration, and evaluation method before launch. Do not stop simply because one version is temporarily ahead.
Privacy and consent requirements vary by location, audience, and implementation. Review the settings and responsibilities associated with your analytics and testing tools, and avoid collecting information that is not needed for the experiment. A fast, accessible page is also part of test quality: if one version loads differently or fails on mobile, the result may measure a technical problem rather than a persuasive change. See the one-page accessibility checklist and the guide to improving landing page speed before launch.
Use this experiment log for every test:
| Field | Record |
|---|---|
| Experiment ID and date | A short name, owner, launch date, and planned review date |
| Hypothesis | Because [evidence], changing [element] from [A] to [B] will [expected effect] for [audience] |
| Primary metric | One conversion event and its exact counting rule |
| Baseline and target | Control rate, minimum worthwhile effect, and expected variation rate |
| Traffic plan | Audience, allocation, estimated visitors, and duration |
| Result | Visitors, conversions, rate, quality checks, and notable segments |
| Decision | Ship, continue, keep control, or create a follow-up test |
Worked examples
Suppose a product launch page receives 1,200 eligible visitors per day. Its control has 48 conversions from 1,200 visitors, for an observed rate of 4%. The team believes a new headline could produce a meaningful improvement to 4.8%, an absolute increase of 0.8 percentage points. They record the hypothesis, keep the CTA and form unchanged, and plan equal allocation.
If the planned total is 12,000 visitors, the initial allocation is 6,000 visitors per version. At 600 eligible visitors per version per day, the traffic estimate is 10 days. The team should still confirm the required sample with its testing platform or statistical method, inspect data quality, and avoid ending the test early if the variation leads for only a few days.
For a lower-traffic local-services page, assume 300 eligible visitors per week and a control rate of 8%. Testing five small changes at once would make it difficult to attribute an outcome and could leave each comparison with too little evidence. A better sequence might begin with the lead-form promise, then test the number of fields, then test the CTA wording. The page owner records each result so that later tests do not repeat an earlier question.
Choose test subjects according to the main uncertainty:
| Element | Useful question | Guardrail |
|---|---|---|
| Headline | Does a clearer outcome improve qualified action? | Keep the promise accurate and specific. |
| CTA | Does different wording reduce hesitation? | Keep the next step obvious and consistent. |
| Form | Does removing fields increase completed submissions? | Preserve information genuinely needed for follow-up. |
| Layout | Does moving proof or the CTA improve comprehension? | Check mobile behavior and accessibility. |
| Pricing display | Does clearer price context reduce uncertainty? | Do not hide material terms or conditions. |
| Social proof | Does relevant evidence increase confidence? | Use verifiable, representative evidence. |
When reviewing the result, compare conversion rates using the same eligibility rules and date range. Look for implementation errors before interpreting a difference. Segment results only when the segment was planned or there is a clear diagnostic reason; repeatedly slicing the data can create misleading patterns.
When to recalculate
Recalculate the plan whenever the inputs change. Revisit the control rate after a major traffic-source shift, campaign change, pricing update, seasonal change, redesign, tracking change, or form change. A new audience may have a different baseline, making the original sample-size and duration estimate inappropriate.
Review the experiment log at the end of every test and at least before each new campaign. Mark whether the result was conclusive, inconclusive, or invalidated by a technical issue. If the test is inconclusive, do not treat that as proof that the versions are equal. Decide whether to collect more suitable traffic, test a larger difference, improve the conversion event, or stop pursuing the question.
Use this action sequence:
- Confirm the primary conversion and baseline from a defined period.
- Write one hypothesis and select one main change.
- Set the minimum worthwhile effect, sample plan, duration, and stopping rule.
- QA both versions on mobile and desktop, including forms, tracking, consent, accessibility, and speed.
- Launch with stable allocation and monitor for technical failures, not just early performance.
- Review only after the planned evidence is available, then record the decision and next question.
For broader campaign planning, pair this workflow with the product launch landing page workflow. For ongoing organic acquisition, use the one-page SEO checklist. Keeping the experiment log current turns isolated A/B tests into a practical record of what your landing page audience responds to and why.