A/B Testing is extremely important if you’re looking to increase your conversion rate. Not just that, if you’re running Apple Search Ads, you need to constantly do A/B Testing in order to improve your Tap Through Rate (TTR).
What is A/B Testing?
A/B testing involves uploading multiple versions of an asset (icon, screenshots etc) to see how users react to different versions of the asset. Finalizing an asset depends on the highest Conversion Rate (CR).
It’s basically experimenting with different versions of something and choosing the one that offers you the highest number of installs.
What all can you A/B test?
In the App Store, you can do different variation testing on the App icon, the multiple App screenshots, screenshot placements, the description, title, sub-title, and the preview video.
How to setup A/B Testing on the App Store?
It’s both simple and complicated. The setup needs to be done via your App Store Connect account. You can only choose to do A/B Testing on the Screenshots and the video from the App Store Connect account. You can add all different variations which Apple calls “Treatments”, allocate traffic to it in percentage, and let Apple show you how users react to it. You can run a test for 90 days. For the App icon A/B testing, you will need to upload a new binary with the different versions and send it to Apple for approval.
How to do A/B Testing on the App Store Icon?
This part is slightly complicated. All variations of the App icon need to be uploaded as a part of a new build and then should be sent to Apple for approval.
Here’s a quick video explaining the process:
Here’s a great Slack discussion explaining the technicalities:
How to do A/B Testing on the Google Play Store?
A/B testing on Play Store is fairly easy. Here’s the official Google guide to it. Login to your Play Store, go to the “Store Presence” tab and then click on the “Store listing experiments” tab. Create a new experiment to run A/B testing. Remember, it is always a good idea to divide traffic between 3 parts. One current asset and the other two would be your new assets.
Give it at least 14 days to gather data. Again, based on the data, make the final choice and stick to it. Give another 15 days to compare conversion rates.
How to make the final decision?
Remember, the data never lies. What users are telling you is what you should go with! Even if your head and heart tell you otherwise. Go with the data. Period.
Some KPIs to take care of when making a decision:
- Visitors
- Installs
- Improvement
- Engagement
- Time to Install
