A Single A/B-test Improved Our Helpdesk App Revenue by 25%
Feb 5 2015 :: by Alex
A lot of our readers are bootstrapped businesses and self-funded startups just like us. So I hope it's OK if we share our findings and challenges from time to time.
These are the results of a recent AB-test we ran on our "buy now" pages. Nothing new or special, just a trivial hypothesis and variations, but the results were overwhelming. Here we go:
Old "buy now" page
Hypothesis: people are distracted by lots of secondary stuff below the main pricing table: the upgrade prices, special deals and bundles, an extra invitation to try the trial version, link to the "billing FAQ" etc. etc. Lets get rid of it to keep them focused.
New "buy now" page
No "upgrade" options, no "trial", no "what happens next" block, no link to the FAQ, and even no footer menu!
- Total visitors tested: 5106
- "A" conversions: 3.91%
- "B" conversions: 5.25%
- Statistical significance: 99%
5.25% to 3.91% might seem like a very small improvement, but it actually means 35% more orders.
Ingredients of a successful AB-test
You probably know that most of your A/B tests will fail. So keep these simple things in mind when running a test:
- Every test should have a hypothesis behind it. Never use the "lets do X and see what happens" formula. Do not test stupid basic nonsense like "lets change the CTA button color from red to green and see what happens" or "lets change the heading font size and see what happens". Always have a hypothesis like "my call-to-action button is not clearly visible, we need it to stand out".
- Wait 2-3 weeks minimum and make sure the results are statistically significant.
- Don't run tests on low-traffic pages - we had to cheat on this one actually, to maximize the number of visitors. While the helpdesk ticketing app is Jitbit's flagship product, we do have a number of secondary products we're selling, for instance, the macro recorder. So we ran the test on ALL the "buy now" pages of our website, not just the helpdesk one. The pages' UI-design is exactly the same, so I think this hasn't distorted the data significantly
- Don't test micro-changes and never trust your intuition even "common sense" logic needs to be tested.
by Alex. CEO, founder