The whole point of testing is to fail a bunch of times. However, if you don’t learn from your failures, you’re foolish because you are missing out on THE most important aspect of A/B testing. Learning. If you are lost at your point I suggest you read up on our introduction to optimization via a/b testing. And a bit more detailed guide on how to split test.
Boss: “Hey you! We want to start running lots of A/B tests starting tomorrow. Make sure you know how to a/b test, and fast!”
You: “Um.. Uhhhh, yes sir!”
You’ve come to the right place. Boss will be very happy that you came to the experts. We will teach you how to a/b test so you can get one up and running by tomorrow! You will become a conversion rate optimization guru.
How to set up your first A/B test
1. First steps… Visit optimizely (it will open in a new window) and view the pricing so you know upfront what the costs will be.
You can start with the lowest bronze plan and always upgrade later. 2,000 visits should be enough to start testing on your site. Believe me when you start seeing results, you will want to test more and see why it’s worth every penny. Great thing about optimizley is that you don’t need to sign up for anything and still use their tool to test out changes on your site.
There is a reason so many people are signed up with optimizely. The secret is that you can turn your investment into tens of thousands of dollars! This isn’t a get rich quick scam, it’s optimizing your site, and improving the user experience for your customers while minimizing your risks. It’s a science.
There are some guidelines you should live by when running A/B tests. Here are some to help you along the way. The do’s and don’ts of A/B testing.
Do get ideas from anyone willing to give it. It’s not up to one person to come up with all the ideas. 2 heads are better than 1, 3 better than…
Don’t dismiss ideas simply because you think they won’t work. If you knew the results of A/B tests without running them, you wouldn’t need to run them. More times than not, the results will surprise you. And it’s only in looking back at the test you will begin to see why the results actually make sense.
This article is geared towards people with homegrown A/B testing platforms. If you use an outside service like optimizely.com, you don’t need to worry about this. They probably have a bunch of tests already that make sure the system is working properly at all times.
However, if you have built your own tools, it’s important that you test your system from time to time. You need to make sure assignments are executing properly at the expected weighting, and that you are recording your data properly. There have been times when using a home grown A/B platform that we’ve run into some troubles.
Which test do you think won?
Take a moment to ponder about which test won and why. I’ll wait.
What is A/B Testing?
Simply put, A/B testing, otherwise known as split testing, is the practice of showing two or more different versions of a page to your users to determine which page performs better. You may be wondering what these different versions may be. Here is a real world example that we’ve tried in the past.