5 A/B testing resolutions you should embrace in 2017
A year’s beginning is the most ambitious of times for teams and businesses — lots of reflection, planning, strategizing, revamping approaches, right? To get you off to a good start, we have listed five A/B testing resolutions that should take in 2017.
A/B testing is a relatively young practice, yet it only takes a Simple A/B Testing Guide and an intuitive A/B testing software to set up the basic process. But it takes a little more deliberate obsession and expertise to squeeze out meaningful results and make decisions that can translate website experiments into business revenue.
In an A/A test, the same page is tested against itself. A/A tests are run to check if an experiment is set up right and if it’s working. It’s concluded that only if the variations “A” and “A” in the test get an equal response from visitors, the experiment is running all right.
A/A tests are a waste of precious time because they require the same amount of traffic, time and resources that an A/B test requires. Take this example: let’s say the average purchase cycle on a website is four weeks, and the experiment period is two or three weeks. In this case, you might lose visitors to the variation that’s not at its best.
There are a plenty of other effective things one can do to make sure the experiment is running fine — cross-browser testing, device testing, user testing, cross-checking with another source of analytics, and of course, obsessively monitoring the experiment to spot glitches.
But if you still insist on an A/A test, then you could try an A/B/A test. That is, you will run an A/A and an A/B at the same time, and in the same experiment. This will require a little more time and traffic than the usual A/A test because it involves three variations, and traffic routed to each will be smaller.
But once you’ve begun, you will probably agree that a sound QA, cross-referencing of analytics and obsessive monitoring (all together) can prove more efficient than an A/A test.
As exciting as it is to run tests and pick winners, unfortunately, a vast number of these tests results are inconclusive. A lot of time, effort and resources go into each experiment — so it’s simply not wise to scratch off an inconclusive test.
Here are a few things you could try differently this year if your A/B experiments are inconclusive:
Your experiment does not end when it reaches a meaningful statistical significance (considering you had the right sample size and an appropriate duration for the test). If you go just by the statistical significance, you might not see results translating into business revenue.
A test should run at least for one entire business cycle, which includes things like different days of the week, diverse traffic sources, your blogs and newsletter schedule or any other external events.
Even after having run the test for an entire business cycle, it’s only smart to repeat the test after an appropriate interval (maybe a few weeks or months, depending on your business cycle). It’s only pragmatic to assume that there are going to be a lot of external influences on the test which might change from one business cycle to another — like the marketing strategy, the targeted audience, a season of the year and so on..
A/B testing tells you which variations clicked with your customer and those that didn’t. An A/B test can only go so far. It cannot tell you what your visitors didn’t like, or why they chose one variation over another, or if your winning variation still has a scope for improvement. The answers to these questions can only come from your visitors themselves.
Polls and feedback sessions on your websites can help you trigger action-based polls now and then to understand the reason behind visitor actions.
Here’s an example of how it can work:
- Use A/B tests to figure what pages are faring well with visitors in terms of conversions.
- Simultaneously run polls to understand the reason behind visitor actions on experiment pages.
- When you determine a winning variant, make changes to the elements that favored the winning variant.
- After you’ve made the change, ask your visitors for feedback and encourage them to express their opinions on the changes you made.
When analytics and user feedback go hand-in-hand, the data and results from your experiments gain additional value.
5. Run tests for mobile visitors exclusively
It’s easiest to run an A/B test on your entire traffic all at once, but that’s not going to give accurate results for many reasons.
One last thing: Test every single day.
Unless you have a real reason for not running a test, you should be running one. When you have zero tests running, it means that you are losing that traffic to “not testing.” And you never know if you’ll have that traffic when you are ready to launch a test. In website experimentation, traffic is like oxygen: you cannot afford to exhaust it without using it.
A Chinese proverb reads, “The best time to plant a tree was 20 years ago. The second best time is now.” So, don’t regret the time and traffic you lost — start now and let A/B testing bring you plentiful conversions in 2017!
From our sponsors: 5 A/B testing resolutions you should embrace in 2017