A/B Test Tool
A/B tests allow you to compare two slightly different versions of something to see which performs better. You can easily run A/B tests on your contribution forms using our platform. Our A/B Test Tool randomly splits everyone who clicks on the link to your contribution form into two groups. One group receives the normal form (the control), and one receives the altered form (the variable). Once the test has run its course, you can compare the two versions and see which one received more dollars and donations.
One example of a good A/B test to try is comparing a plain pop-up recurring ask with a pop-up recurring ask that includes an added incentive for those who sign up, like a bumper sticker. We’ve found that this is one of the best ways to get recurring donations.
Setting up an A/B test
To set up an A/B test, click the "Edit" button to the right of your form's name in the Contribution Form tab to open the form editor.
Select "A/B testing" in the form editor.
Click the link.
You'll be taken to the A/B Test tab of the Form Tools menu where you can set up your experiment.
Give your test a descriptive name. The name will only be seen internally, so choose something that works best for you.
Underneath the name field, the “Automatic traffic allocation” option will automatically be selected (pictured above). This feature, also known as a multi-armed bandit algorithm, directs more traffic to the version of your form that is performing best so you won’t lose out on potential contributions, while still running a sound test. This is an easy, and highly recommended, way to run tests and raise funds at the same time. You can learn more about the multi-armed bandit on our blog.
If you uncheck the “Automatic traffic allocation” box, both of your forms will be allotted the same volume of traffic, even if one performs very poorly.
Select the feature that you would like to test:
- Form title
- Contribution form pitch
- Branded layout
- Fundraising thermometers
- Embedding a video
- Using a countdown
- Pop-up recurring title
- Pop-up recurring ask
This will be your variable. While you can test multiple features at once, if you want to run the most accurate test possible, you should pick just one. That way you will know it was the variation that caused a difference in visits, contributions, or contribution amounts.
Once you’ve selected which feature you want to test, update the variation of that feature as needed. For example, if you are testing the form title, your current form title will populate and you’ll need to enter a second variation of the title (pictured below).
Once you’ve finalized your test variations, click the blue “Create Test” button. Congrats — your test is live!
You will now be able to view both variations of your form. Go to your form and open the form editor. In the "A/B testing" section, select "control" or "test" from the dropdown menu.
TRACKING YOUR A/B TEST
Once you create the test, you’ll be able to track each variation's progress by returning to the form editor for your form and going to the "A/B testing" tab. Then follow the link.
You'll see the traffic allocation, conversion rate, number of people who donated (conversions), number of visits, average contribution, and dollars per visit for each version of the form.
If you did NOT use the automatic traffic allocation (multi-armed bandit) option, you will need to click “Make Winner” on the variation that performed best in order to stop the less-successful form from being distributed. If you did use the multi-armed bandit method, the A/B test will do this on its own over time, but if you see a clear winner you can click “Make Winner” to speed up the process.
Data on your A/B test will be included in your contributor CSV, which can be found on your form's Statistics page in the “Additional tools” dropdown near the bottom. You can get to the Statistics page when you are viewing your form by clicking on the Stats button in the form menu on the left, or you can get to the page by clicking the Statistics tab in the Form Tools menu if you are not viewing your form.
If you've offered any additional incentives, like a bumper sticker on a pop-up recurring test, information for all of the donors that signed up will be in the CSV as well.
If you have further questions about A/B tests or best practices, let us know.