How to Do A/B Testing

Chapter 5

How to A/B Test Your Landing Pages and Improve Your Conversion Rate

A/B testing, also called split testing, is like a mini-experiment that helps you improve your conversion rate by helping you better understand what compels your audience to take action.

Does your audience want a red or yellow CTA button? Do they click more when it’s $5 or 10% off their purchase? Do they want long-form or short-form content? Does a video make them more likely to convert?

Landing page A/B testing helps you determine which elements of your page have the biggest influence on your conversion rate. It allows you to fine-tune your campaign to ensure every single element on your page is optimized for conversion and targeted to your specific audience.

Keep reading to learn how to run an A/B test and why it’s so important for conversion rate optimization.

What is A/B testing?

A/B testing lets you create two versions of a landing page or webpage and then publish both and review the analytics to see which has a higher conversion rate. Half your traffic will be directed to one version, and the other half will be directed to another. Once you’ve identified ‘the winner,’ you can direct all your traffic to only your highest-performing content.

To get the best results, your A/B test should compare one variable at a time, such as a headline, color, font, image, or layout. (Multivariate testing, on the other hand, looks at multiple variables at once, but we don’t typically recommend this for our clients).

For example, you might have one version with a red CTA button and another version with a yellow button. If one performs better than the other you know the color of your buttons influenced your conversion rate.

Why do we do A/B testing?

A/B testing is about understanding your target audience. As we like to say, “guess less to grow more!”

It’s a strategy to understand your customers’ preferences and behaviors, so you can appeal to them in a way that promotes conversion.

Be honest with yourself: How well do you know your target audience?

What colors resonate the most with them?
Do they like long-form or short-form pages?
Do they prefer videos, pictures, or content?
What kind of language and tone do they interact with?
Do they prefer email, phone, or social media?
Do they like dogs or cats?

If any of these questions left you feeling like you don’t know, then A/B testing is the perfect marketing tactic for you. It will help you answer these questions (or more relevant ones), so you can design your landing pages with your audience top of mind.

When you know what your audience wants, you can create a page optimized for conversion.

Single variable A/B testing

We almost always recommend that you test one variable at a time. If you change too many variables at once, you won’t know where any changes in engagement are coming from.

Decide which variable you’ll change and for how long you’ll run the test. When the timeline is over, the “winning” page with the higher conversion rate stays. Then you use that page to test the next variable.

It’s like building a page brick by brick. It takes patience, but you’ll start to see your conversion rate ramp up in no time.

A/B testing might sound complicated. But it’s actually easier than you think. Here are the steps for running your A/B testing:

Create a hypothesis. For example, “Red buttons will perform better than blue buttons.” (Don’t get attached to the outcome of the hypothesis.)
Define two variations based on the hypothesis. For example, red vs blue.
Divide incoming traffic. Don’t divide traffic forms, like your Adwords go to red and your social media go to blue, as this could skew the results. Try to split each traffic route in half.
Run the test for a statistically significant time period. This could be anywhere from 2 weeks to 6 months.
Find the winner with the higher conversion rate. Send all of your traffic there.
Replace or delete the losing version.
Repeat with a new variable.

The benefits of conversion rate optimization

How to perform an A/B test with Leadpages

Unlimited A/B testing is available to Leadpages Pro and Advanced customers. Best of all, you can set up and complete your test in just five steps:

On your Leadpages dashboard, find the page you want to test. In the page's "Action" menu (•••) click “Create new split test.’
Add a variation by creating a copy of your control, or choose a page you’ve already built.
Set the traffic distribution (in most cases it will be 50/50).
Review and launch your test.
When you have your results, you can simply disable the non-winning page and direct all traffic to the winner.

How long should you run your A/B test for?

We recommend running a single test for at least two weeks, to make sure you have sufficient data to draw a conclusion. However, this will depend on your typical incoming traffic and the length of time you have to work with.

For example, lower-traffic pages will need longer to generate a significant sample size. On the other hand, a page with a high amount of traffic might produce meaningful results in just a few days.

A/B testing examples

So which variables should you test? Here are the most popular elements to change and test.

1. Headlines

The headline is your first impression and your first chance to sell. It can intrigue people to read more or instantly send them away. It’s a make-or-break moment, so don’t be afraid to test and retest to make sure you’re resonating with the largest possible audience.

Test different lengths, characters, or phrasing. Try more emotional words, or rewrite the whole headline with a different tone. Add an exclamation point. Remove a word. Little changes can make a huge difference.

The goal of your headline is immediately show the benefits of your offer and compel the viewer to take the next step (like make a purchase). In general, you want your headline to be:

Succinct and clear
Evocative and stimulating
A representation of your main value proposition

2. Page layout/template

The layout of the page can make a huge difference in how your audience engages with the content. Test out different layouts to see how your visitor interacts with the information and CTA, based on where it is on the page.

Check out our library of over 250 landing page templates to see examples of different page layouts.

3. Hero image

Just like your headline, the hero image has a big impact as it’s the first visual element visitors see. Images stimulate an emotional response and subconsciously send signals to your visitor. Try a stock image versus an illustration versus original photography to see which resonates with your audience. Test the difference between product photos, lifestyle images, or a “human” photo of your employees or customers.

4. Form fields

You want as little friction as possible at the conversion point. That means you should minimize the number of fields on your opt-in form so the visitor can move faster through the conversion.

If you’re collecting leads, try to limit the form to just a first name and email/phone. You can always request more information from them later, but the primary goal now is to just get a means to contact them in the future.

That all being said, more information from your leads and customers is valuable. Add and remove form fields to see how this impacts your conversion rate and decide if having extra fields is worth it.

5. Call to action (CTA) button

The CTA button is your salesperson that pulls the lead through to the next step.

Here are a few questions to ask yourself:

Is your CTA button the right color? Test different colors to find out.
Does it use the right copy to compel people to click? See if using first-person language works better than third-person. Test if they prefer direct language or a goofy and funny tone.
Is the CTA positioned in an effective part of the page? Can they see it (and click on it) whenever they’re ready to convert?

A lot of our users see the biggest results after A/B testing and optimizing this button. This is the point where they’re converting, so you want to really emphasize optimization here.

6. Density and length

Do you need a lot of words to explain your offering? Or do your visitors prefer short, sweet, and to the point? Try out different copy and content lengths to see what grabs your audience’s attention. You want to give them just enough information to be able to make the decision to convert.

7. Numbers

The “numbers” on your page work the hardest to convince your visitor to convert. The more specific your numbers are, the better.

For example, “$100 off” might be more effective than “10% off purchase price,” but you won’t know until you test. Different audiences react to discounts, numbers, and statistics differently. So try out different ways of presenting your numbers to see what most resonates with your customer.

8. Testimonials

Customer reviews and endorsements increase the perceived value and credibility of your offerings. They also help customers envision their own lives after they’ve converted. Does the visitor see themselves in the people giving testimonials?

Not all landing pages need testimonials. In some cases, they might add noise and distract from the CTA. Run some A/B testing to see if testimonials are convincing people to convert or detracting from the main point.

If testimonials or reviews are too long, you can also consider adding links to your social media accounts (if you have a strong presence). This minimizes the friction on your page while adding in some social proof.

9. Offer

Sometimes it’s the actual offer or conversion action that’s deterring visitors. You might want to consider adjusting your price point, changing the time or day of your event, rewording the title of the book, or other adjustments to the offer itself.

You can run an A/B test to see what kinds of offers your customers want. But you should also ask them directly. How do they want to receive value from you, and what will they give in exchange for that value? Don’t be afraid to ask for feedback.

10. Progress Bar

Behavioral inertia is real. Once you have a visitor moving in any given direction—towards conversion or clicking away, they’re likely to keep moving that way… unless they run into an obstacle or distraction.

Showing them a progress bar is a great way to keep them moving forward. They’ll see that they’re almost done with the conversion, so they’ll keep at it until the end. We’ve seen progress bars dramatically increase the number of people who finish a conversion action.

Frequently asked questions about A/B testing

How many visits does my landing page need before my A/B test is valid?

The number of visits needed for an A/B test to be valid depends on several factors, including the desired level of statistical significance, the expected conversion rate, and the minimum detectable effect (the smallest difference you want to be able to detect between the variations). As a general rule of thumb, you should aim for at least a few thousand visits per variation to ensure you have enough data to reach a statistically significant conclusion.

What should I do if my A/B test results are inconclusive?

If your test results are inconclusive, consider these steps:

Review Your Hypothesis: Ensure your test was based on a strong hypothesis.
Check Sample Size: Make sure you had enough traffic and data for reliable results.
Analyze User Feedback: Gather qualitative insights from users to identify potential issues.
Iterate and Retest: Make adjustments based on your findings and run another test.

What common mistakes should I avoid in A/B testing?

Common mistakes in A/B testing include:

Testing Too Many Variables at Once: Focus on one variable to isolate its impact.
Ending Tests Too Early: Ensure you run tests long enough to reach statistical significance.
Ignoring User Segments: Different user segments may respond differently to variations.
Not Having a Clear Hypothesis: Define a clear, testable hypothesis before starting.

Neglecting Post-Test Analysis: Analyze not just the winning version, but also why it won and any patterns in user behavior.

What’s next?

Now that you have all the knowledge and tools you need to be a conversion rate optimization expert it’s time to put it all together. In Chapter 6 we go over how to create a conversion-optimized website or landing page.