A/B Test Tool
A/B tests allow you to compare two slightly different versions of something to see which performs better. You can easily run A/B tests on your contribution forms using our platform. Our A/B Test Tool randomly splits everyone who clicks on the link to your contribution form into two groups. One group receives the normal form (the control), and one receives the altered form (the variable). Once the test has run its course, you can compare the two versions and see which one received more dollars and donations.
One example of a good A/B test to try is comparing a plain pop-up recurring ask with a pop-up recurring ask that includes an added incentive for those who sign up, like a bumper sticker. We’ve found that this is one of the best ways to get recurring donations.
Setting Up an A/B Test
To set up an A/B test, click the "Edit" button to the right of your form's name on the Contribution Forms page of your Dashboard. This will open up the form editor for the form you’d like to A/B test.
Select "A/B testing" in the form editor.
Click the link in the “A/B Testing” tab.
You'll be taken to an A/B Test page, where you can set up your experiment.
Give your test a descriptive name. The name will only be seen internally, so choose something that works best for you.
Next, add a short hypothesis that describes what you are looking to find out with your A/B test.
Underneath the “A/B test name” and “Hypothesis” fields, the “Automatic traffic allocation” option will automatically be selected (pictured above). This feature, also known as a multi-armed bandit algorithm, directs more traffic to the version of your form that is performing best so you won’t lose out on potential contributions, while still running a sound test. This is an easy, and highly recommended, way to run tests and raise funds at the same time. You can learn more about the multi-armed bandit on our blog.
If you uncheck the “Automatic traffic allocation” box, each of your variants will be allotted the same volume of traffic, even if one performs very poorly.
Select the feature that you would like to test:
- Form title
- Contribution form pitch
- Branded layout
- Fundraising thermometers
- Embedding a video
- Pop-up recurring title
- Pop-up recurring ask
- Preset amounts
This will be your variable. While you can test multiple features at once, if you want to run the most accurate test possible, you should pick just one. That way you will know it was the variation that caused a difference in visits, contributions, or contribution amounts.
Once you’ve selected which feature you want to test, update the variation of that feature as needed. For example, if you are testing the form title, your current form title will populate and you’ll need to enter a second variation of the title (pictured below).
Once you’ve finalized your test variations, click the “Create Test” button. Congrats — your test is live!
You will now be able to view both variations of your form. Go to your form and open the form editor. Go to the "A/B testing" tab and select which variation of the form you would like to see from the “Preview A/B test variation” dropdown menu.
Tracking Your A/B Test
Once you create the test, you’ll be able to track each variation's progress by returning to your form, opening the form editor, and going to the "A/B testing" tab. Then follow the link.
You'll see the traffic allocation, conversion rate, number of people who donated (conversions), number of visits, average contribution, and dollars per visit for each version of the form. You will also see the name of the A/B test, the hypothesis, and which admin at your campaign or organization started the test at the top of the page.
If you did NOT use the automatic traffic allocation (multi-armed bandit) option, you will need to click “Make winner” on the variation that performed best in order to stop the less-successful form from being distributed. If you did use the multi-armed bandit method, the A/B test will do this on its own over time, but if you see a clear winner you can click “Make winner” to speed up the process.
Data on your A/B test will be included in the CSV detailing your form’s contributions, which can be found on your form's Statistics page in the “Additional tools” dropdown near the bottom.
You can get to the Statistics page when you are viewing your form by clicking on the "Stats" button on the form navigation bar.
You can also get to the Statistics page by clicking either the number of contributions or the donation total for your form on the Contribution Forms page of your Dashboard.
If you've offered any additional incentives, like a bumper sticker on a pop-up recurring test, information for all of the donors that signed up will be in the CSV as well.
If you have further questions about A/B tests or best practices, let us know.