In our last blog, we presented the scope of A/B Testing, what it is and why digital platforms should focus on A/B Testing. In this blog, we will talk about how to perform A/B Tests.
How to perform an A/B Test?
A/B testing allows systematic way to find out what is working on a digital platform and what isn’t. Driving traffic to a digital platform is hard enough, and therefore providing the best digital experience to maximize the chances of conversion is of paramount importance. A/B Testing allows digital platforms to maximize conversions and identify the issues hindering conversions. It is important for digital platforms to create a structured and continuous A/B Testing plan, and not look at A/B Tests as a one-time activity. The steps in A/B Testing involves:
1. Research: Before initiating an A/B Test, it is important to create benchmarks i.e. how the platform is performing currently. Data points related to user visits, most visited pages, conversion goals of each page etc. The clicks and browsing behavior using standard heatmap tools can provide insights on time being spent on different sections of a page
2. Hypothesis Design: The next step then is to define a hypothesis aimed at increasing conversion. A sample hypothesis could be: A title having the product USP (unique selling point) leads to increased clicks on ‘View Details’ button
3. A/B Test Cases: The two versions of the webpage – based on the hypothesis in the previous step need to be created. Continuing with the previous example, the existing convention of product title is the ‘control’ and a page with title having the product USP would be the variation
4. Run Tests: Launch the test and let visitors generate sufficient data to help you arrive at statistically significant results.
There are primarily four types of testing: A/B Testing, Split Testing, Multivariate Testing and Multipage Testing. You need to identify the right test based on your experiment goals.
5. Analysis & Conclusion: Once sufficient data has been generated, analysis of results allows you to arrive at a data-driven conclusion i.e. which variation of the test is better. It might be possible that the test is inconclusive, in which case you would need to learn from the test and implement changes so that subsequent test(s) can provide clear winners
Mistakes to avoid while A/B Testing
1. Invalid or poor formulation of hypothesis: A/B Test case can only be created against a hypothesis you want to test. A poorly/wrongly formulated hypothesis will take you nowhere
2. Testing multiple elements in one A/B Test: A/B Test is mostly used to test one variation at a time – so that the differences can be measured. Too many variations may lead to ambiguous results
3. Not measuring statistical significance: The difference between two versions should be statistically significant to make the conclusions actionable
4. Unbalanced Traffic: Traffic should be balanced and not biased towards a certain type of visitors
5. Running a test for insufficient duration: Running an A/B Test for inappropriate time may result in insignificant conclusions
6. Accounting for external factors: A/B Tests should be avoided on days with higher traffic, holidays etc. to avoid skewed results
A/B Tests, if used well, can significantly improve the return on investment (RoI) of marketing campaigns. It helps the digital platform owners identify existing problems, address them and reach towards the desired conversion goals