Jargon Buster / A/B Testing
A/B Testing.
Ads, Data & DemandShowing two versions of a page, ad or email to similar groups to see which gets more people to act.
What is A/B testing?
A/B testing is a controlled experiment where you show two versions of the same thing, such as a landing page, an ad headline or an email subject line, to comparable groups of people and measure which version produces more of the action you care about. Version A is usually what you have now. Version B changes one thing. Visitors are split at random, so any clear difference in results can be put down to the change rather than to luck or timing.
Why it matters
Every website and campaign is built on guesses about what people want. A/B testing swaps some of those guesses for evidence. For a business paying for clicks, a page that turns more visitors into enquiries lowers the cost of every lead without adding a penny to the ad budget. It also settles internal arguments: instead of debating whether the button should say 'Get a quote' or 'Book a call', you let customers decide.
What a fair test needs
- One clear hypothesis Write down what you are changing and why you expect it to help. 'Showing prices will cut time-wasting enquiries' can be tested; 'make the page better' cannot.
- A single primary metric Decide in advance whether you are judging on enquiries, sales or sign-ups. Choosing the metric after the results arrive is how people fool themselves.
- Enough conversions Small samples swing wildly. If a page produces a handful of enquiries a week, a test may need months to say anything reliable.
- A fixed end date Run tests for whole weeks so weekday and weekend behaviour both count, and stop on the date you set, even if an early result looks exciting.
The honest view
The biggest obstacle to A/B testing in small and mid-sized businesses is volume. Many SME websites simply do not get enough traffic to detect small improvements, so tests on button colours or minor wording either end inconclusive or, worse, crown a false winner that someone then celebrates. Be wary of any result that arrived quickly.
The tests worth running at SME scale are bold ones: a different offer, a different headline promise, a shorter form, prices shown or hidden. Big changes create differences large enough to measure.
What to do about it
Pick the one page that matters most commercially, usually your main landing page or quote form, and check how many enquiries it produces each month. If the number is modest, test only bold changes, or test on paid traffic where you control the volume. Write a one-line hypothesis, choose one metric, set an end date and record every result in a shared log, win or lose. Email subject lines make a good first test.
Related terms.
Conversion Rate Optimisation (CRO)
The ongoing work of increasing the share of website visitors who take a valuable action, such as an enquiry.
Brand, Web & DesignSee more LLanding Page
A focused web page built for one audience and one action, usually the destination for an ad or campaign.
Brand, Web & DesignSee more KKPI (Key Performance Indicator)
A measurable number chosen because it shows whether you are moving towards a goal that matters.
Ads, Data & DemandSee more GGA4 (Google Analytics 4)
Google's free analytics platform for measuring how people find and use your website or app.
Ads, Data & DemandSee moreGot a question about your own setup? Let's talk.
Your first call is with our founder, Rhodri Williams.