What is landing page A/B testing and how does it work?
A/B testing (sometimes called “split testing”) is a marketing strategy that pits two slightly different variations of the same page against one another to see which one yields the highest conversion rate.
For example, while you’re building your landing page you might wonder how big your call-to-action (CTA) button should be. Or if a different headline would convert better. Or would an on-page countdown timer help or hinder your sign-up rate? For all these questions and others like them, following your intuition or web design best practices will only get you so far. To discover the truth of what really works, you need to approach it like a scientist: you need to test.
The way it works is pretty simple:
- Create a modified version of your landing page containing a single change—the “variable”—that you think could positively impact your conversion rate (a bigger CTA button, different headline, countdown timer, etc.).
- Randomly assign half your landing page visitors to the original version and the other half to the modified version.
- Let the test run until you have an adequate sample size and compare the data from both versions of the page to see if your changed landing page had a positive or negative impact on your conversion rate.
- If the change is positive, you adopt the modified version as your new, go-to landing page. If the change had a negative impact, continue using the original.
It’s important to appreciate that A/B testing isn’t a one-and-done type of activity. It’s an ongoing process that involves making incremental changes to your page in a bid to fine-tune your campaign for maximum conversions. Every test you run builds upon the test that came before it. Even a negative result is useful in that it gives you a better idea of what won’t work when it comes to designing your next test.
You’ll often hear A/B testing being mentioned in the same breath as multivariate testing. The two methods are quite similar but have some important differences. While A/B testing compares two different versions (A and B) of a landing page by changing one variable at a time, multivariate testing instead compares multiple variables at a time.
10 AI Lead Generation Strategies to Boost Your Conversions
Download the free guide to discover AI lead gen strategies that will automate your marketing, cut your costs, and deliver better leads—fast.
Why do we do A/B testing?
There’s an old military saying that goes something like this: “No plan survives first contact with the enemy”. The idea is that no amount of planning can fully predict how things will pan out when your plan is eventually put into action.
The same basic logic applies to your landing pages (although we don’t advocate thinking of your visitors as enemies!). Even your best guess of what will make your visitors most likely to convert is, in the end, just a guess. There are likely factors you haven’t considered and what you thought would work might pan out differently in practice. That’s why roughly 60% of companies perform A/B tests on their landing pages.
The beauty of A/B testing is that it ties the success of your sales and marketing campaigns to hard data instead of mere guesswork. By continually testing your assumptions, you end up discovering exactly what drives your audience to take action so you can keep improving the conversion rate of your landing page.
With that in mind, here are some major benefits of A/B testing:
Enhanced user experience
When someone visits your landing page, chances are they have a specific goal in mind. Maybe they just came to learn more about your product or service, maybe they’ve already decided they want to buy from you, or maybe they’re curious about who you are and what you offer.
Whatever their goal, it’s important to make their experience on your page as frictionless as possible. Confusing copy, jarring color schemes, and hard-to-find sign-up buttons are all potential stumbling blocks that could detract from the user experience and, in turn, damage your conversion rate.
A/B testing helps you identify these kinds of problem areas so you can adapt and create a more free-flowing user experience.
Low-risk, high reward
Since A/B testing is an incremental process, with only single changes being made at each step, the risk of your conversion rate falling off a cliff during any given test is relatively minimal. And even if you do find that changing a certain variable causes a significant dip in your conversions, you can simply change it back and use that knowledge moving forward. And of course, if the change improves your conversion rate, then you’ve found something worth keeping!
Understand your audience better
The discoveries you make by running A/B tests will improve your understanding of what drives your target audience’s behavior. And the better you understand your target audience, the better your future guesses will be when it comes to improving your user experience and conversion rates even further.
You can also use your findings in other areas of your business to further tailor your content, products, and services to your audience’s preferences.
Lower bounce rates
Your bounce rate is the rate at which visitors land on your page and leave without exploring any other pages on your website. It goes without saying that the more pages they visit, the better chance there is they’ll convert. As you’re performing your landing page A/B test, see how the different variables affect your bounce rate and take that into consideration when choosing a winner.
Increase conversion rates
Even small changes can have a massive impact on your conversion rates—sometimes by as much as 300%. That’s why A/B testing is so important. By testing every element of your landing page and finding what drives the best results, you’ll eventually be left with a fully optimized landing page that converts at a crazy high rate.
More Revenue
A lot of businesses create new offers, products, and services in order to increase revenue. But since it takes a fair amount of time and money to generate new, high-quality traffic, it’s much more effective to get the most out of the traffic you already have. By using A/B testing to optimize your sales pages, you’ll increase your conversions which in turn will increase your revenue—without having to come up with any new offers.
Easily create your website and landing pages with the only platform engineered by marketing nerds.
Examples of landing page A/B testing success
If you’re still not fully convinced that it’s worth incorporating A/B testing into your campaigns, perhaps the following stories from some of our customers will sway you.
+28% conversions by removing the CTA
Leadpages customer, The Foundation, discovered that simply deleting the action-oriented headline on their video squeeze page boosted their conversions by 28%. Even more surprising, the Foundation team also ran the same test on their homepage. In both cases, the “no headline” variation won out by a significant margin. How’s that for counterintuitive?
+28% conversions by adding a progress bar
Progress bars often improve conversions because they let people know how close they are to finishing a form or sign-up. But we wanted to prove this, so we set up a split test. The page that used the progress bar had a conversion rate 28% higher than the control variation.
+34.43% conversions by using the right product image
We’ve often noticed that conversion rates increase when you include an image of your product in your opt-in box. One of our customers, Jae Jun at Old School Value tested the performance of two different images of his stock-analysis software package. The more modern-looking of the two images outperformed the more traditional-looking image by 99.76%.
+75% conversions by changing an image
Leadpages user Carl Taylor once split-tested two different images of himself on his landing page. One was him in a suit and tie, and the other was him dressed more casually. To his surprise, the one of him dressed more casually increased his conversion rates by over 75%.
+307% conversions by using the right headline
Award-winning author Amanda Stevens got dramatic results when she ran a simple headline A/B test on her eBook landing page. Her conversion rate skyrocketed just by switching her headline from “New Book Reveals Rescue Remedies for Retailers” to “If you’re a retailer in need of fresh ideas and proven growth strategies, this book is for you!”
A big thanks to all the Leadpages customers and small business marketers who have given us permission over the years to publish the results of their landing page A/B tests and split tests.
How to decide what to test
Okay. So you’re sold on the benefits of landing page A/B testing and want to run your first test. But where do you begin?
It wouldn’t make much sense to just start testing things at random. If you can’t think of any reason why your proposed change would have a positive impact on your conversion rate then there’s no reason to test it.
For quicker and better results, base your hypotheses on the current performance of your landing page. There are multiple ways to gauge how well your page is performing, from analyzing the quantitative data found in your analytics to reviewing the qualitative feedback you get from testing the page with your colleagues or friends.
The basic point is to use all the data sources at your disposal to work out where your page could do better. For example, if your analytics show that 60% of visitors drop off while they’re filling out the sign-up form, maybe you should reduce the number of form fields. If your friend has trouble understanding what the page is selling, maybe your headline should be clearer.
To help you decide which tests to run first, we recommend using the ICE prioritization framework. This framework poses three questions for you to answer when considering whether you should run a test.
- Impact: How impactful do you expect this test to be?
- Confidence: How sure are you that this test will prove your hypothesis?
- Ease: How easily can you launch this test?
You should answer each question with a score from 1 to 10. The average of these three scores is your test’s overall ICE score.
For example, let’s say you give your proposed test the following scores: Impact: 5, Confidence: 4, Ease: 8. In this case, your test’s overall ICE score would be 5.6. You can then compare this score with other proposed test scores to determine which tests to try out first.
A/B testing ideas for different parts of your landing page
Which variable you decide to modify during any given A/B test will depend on your goals and the unique details of your landing page. That said, modifying any one of the following landing page elements could have a big impact on your conversion rates:
Headline
First impressions are everything and since your headline is the first thing your visitors will read, it’s essential to nail it. A bland, confusing, or irrelevant headline will deter many visitors from staying on the page, thereby damaging your conversion rate.
If you think your headline could use some work, try testing different title lengths, wordings, and benefits/pain points. For example, instead of a generic headline like “Grab this eBook while you still can!”, try something more specific and evocative like “Learn the Success Strategies That Will Propel Your Music Career Forward”.
Crafting the perfect headline is a balancing act. You want it to be compelling enough that people respond to your call to action, informative enough that people understand the value you’re offering, and brief enough that people don’t get bored or frustrated and leave the page.
Copy
As with your headline, your landing page body copy should make it clear what value your visitor will get from handing over their contact details or payment info. Make sure that your copy also addresses any obvious questions and potential objections, and try to strike the right tone for your audience.
Some things you can test with your copy include:
- Different sub-headers.
- The order in which you present your ideas, subjects, or topics.
- The benefits you focus on.
- Formal versus casual language.
Images
The images you use have just as big an impact on your conversion rate as your copy, so don’t overlook them. Generally speaking, images with smiling faces have a positive effect on conversion rates. If you’re selling a product, using pictures that clearly display the features and benefits will also improve your performance.
As with other landing page elements, it’s impossible to know for sure what your audience will respond to, so make sure to test a few different images to ensure you find the right one.
When it comes to opt-in forms, it’s usually best to keep the number of form fields to a minimum. Users may be put off if they think you’re asking for too much information. For most lead capture forms, all you really need is a name and email address—anything else is a luxury.
So, if you have a longer opt-in form try testing it against a shorter one to see if that makes a difference.
Your CTA button is the focal point of your landing page, where you prompt your visitors to take whatever step you need them to take. Given their extreme importance, it’s essential to make sure that your CTA buttons are clear, compelling, and easy to see. Any aspect of your CTA button could have an impact on your conversion rate, from placement to wording to color. Try out different variations to see what works best.
Countdown Timers
Countdown timers are a super useful tactic for creating a sense of urgency when visitors land on your page. The unmistakable, visual cue lets visitors know that the value you’re offering won’t be available for long and they’d better act now if they don’t want to miss out. If you’re promoting a limited-time offer, why not test to see if the presence of a countdown timer boosts your conversion rate?
Social Proof
Adding some credentials, testimonials, or endorsements to your landing page can often improve conversions by increasing the trustworthiness of your product or service. On the other hand, these elements are sometimes unnecessary and can detract from your CTA by cluttering your page. Which is the right option for your campaign? Create a test to find out.
Price Points
Pricing experts have understood the power of numbers for decades. In the minds of consumers, $4.99 seems a lot cheaper than $5, and on restaurant menus, the dish listed as “bruschetta: $5” seems more expensive than the one that appears as “bruschetta: 5.” Try to use these subtle differences to your advantage when testing the way you present the numbers on your page.
Page length
When it comes to landing page word count, the general consensus is that complicated and expensive offers require more text than simple low-cost offers. However, that’s not always the case. Just like everything else we’ve mentioned, the only way to know for sure is to test.
If you’re not sure how long your page should be, create a short and long version and pit them against each other to see what your audience prefers.
How do you perform a landing page A/B test in Leadpages?
So, you’ve decided which element of your landing page you’d like to modify and test. How do you put your A/B test into action?
Fortunately, if you’re already a Leadpages Pro or Advanced user, you can easily run as many A/B tests as you like directly within your account. Here’s how:
- Inside your Leadpages dashboard, click “Create New Split Test.”
- Find the page that you want to run a split test on and select it as your Control.
- Add a variation by creating a copy of your control, or choose a page you’ve already built.
- Select the percentage of traffic you’d like each page to receive.
- Review and launch your test!
For a more detailed breakdown of how to implement A/B tests in Leadpages, check out the video below:
Best practices for A/B testing landing pages
To get real, reliable results from your A/B tests, strategy matters just as much as execution. Whether you're optimizing for conversions, signups, or sales, following these best practices will ensure your tests are focused, accurate, and actionable.
Test One Variable at a Time
Unlike multivariate testing, A/B tests work best when you limit yourself to one or two variables. This makes it much easier to identify which change, if any, led to a positive impact on your conversions. Use the ICE framework mentioned above to prioritize your testing roadmap.
Let Your Test Run Long Enough
How long should you run your A/B test? The answer to this question will vary from case to case, but one of the main factors to consider is how much traffic you receive. As with any statistical analysis, you need to make sure that your sample size is large enough for it to yield conclusive results. Put simply, the lower the traffic you receive, the longer it makes sense to run your test before declaring a winner.
Segment and Personalize When Possible
One test won’t fit all. If your audience segments behave differently—by device, traffic source, or location—consider running segmented tests. You might find that mobile users prefer a more compact design, or that Facebook traffic responds better to certain copy.
Measure More Than Just Clicks
While click-through rate is a common metric, it’s not always the best indicator of success. Track meaningful conversions like form submissions, purchases, or bookings. And monitor bounce rates, time on page, and scroll depth to get a fuller picture of how users interact with your landing page.
Don’t Guess—Keep Testing
A/B testing isn’t a one-and-done process. Even small improvements compound over time. Once you’ve identified a winning variation, use it as your new control and test something else. The brands that win the most leads aren’t lucky—they’re consistently iterating.
Mistakes to avoid when running landing page A/B tests
While the basic principles of A/B testing are simple, putting things into practice can sometimes get a little tricky.
So before we finish, here are a few common mistakes to avoid when implementing your testing campaign:
Failing to account for past results in your testing
To get the most out of A/B testing, it’s crucial that you consider your past findings when deciding what to test next. Keep an up-to-date record of all the tests you’ve run in the past, including any information on what worked and what didn’t. This will give you a much better chance of generating fruitful, data-driven hypotheses and avoiding repeating the same test twice.
Testing the wrong page
Not every page is worthy of an A/B test. Pages like your About page, Contact page, and even your homepage might not be designed for conversions. If a page doesn’t have one clear action that you want users to take it’s hard to decide whether one variation was more successful than another. That’s why you should reserve A/B testing for landing pages with one clear CTA.
Changing the variables mid-test
It’s possible that you come up with other A/B testing ideas after you start your first test. However, that doesn’t mean you should alter your test before it’s finished. Changing the parameters before the test is complete will skew the results, making it difficult to determine what impacted your conversion rate.
Instead, let your test play out and then begin a new test once it’s finished.
Understanding statistical significance in landing page A/B testing
Running an A/B test is easy. Knowing when your results actually mean something? That’s where statistical significance comes in.
What Is Statistical Significance (And Why It Matters)?
Statistical significance tells you whether the difference in performance between Variant A and Variant B is real—or just due to random chance. Without it, you could end up making decisions based on “lucky” results that don’t hold up over time.
Think of it this way: A test might show a 10% boost in conversions, but unless that result is statistically significant, you can’t confidently say Version B is better.
Set Your Confidence Level
In marketing A/B testing, the standard benchmark for confidence is 95%. That means there’s only a 5% chance the results are due to random variation—and a 95% chance they reflect a true difference. If you’re seeing results with 95%+ confidence, you’ve got a winner.
How Do You Know If Your Test Is Significant?
You can use a statistical significance calculator (many are free online). You’ll typically need:
- The number of visitors in each variant
- The number of conversions in each variant
The more data you have, the easier it is to detect real differences. Small sample sizes can lead to false positives—so be patient and let the test run its course.
Pro Tip: Avoid “Peeking”
It’s tempting to check in early and declare a winner based on initial results. But early fluctuations are often misleading. Commit to a sample size or timeframe before launching the test, and wait until the data has matured before making a call.
Time to put A/B testing to the test!
We’ve seen how A/B testing can be an extremely powerful method for reliably improving landing page conversion rates. By helping you pinpoint what works and what doesn’t, A/B testing paves the way for more successful, revenue-generating campaigns.
Now it’s your turn to start putting these ideas into practice. Let this post be your guide and don’t be afraid to make small mistakes—after all, a big part of A/B testing is figuring out what works and what doesn’t work.
Don’t forget that Leadpages Pro and Advanced users can run unlimited A/B tests directly within their accounts. So if you haven’t already signed up, why not start your 14-day free trial today?
10 AI Lead Generation Strategies to Boost Your Conversions
Download the free guide to discover AI lead gen strategies that will automate your marketing, cut your costs, and deliver better leads—fast.
