Using the A/B Test Tool

A/B tests allow you to compare two slightly different versions of something to see which performs better. You can easily run A/B tests on your contribution forms using our platform. Our A/B Test Tool randomly splits everyone who clicks on the link to your contribution form into two groups. One group receives the normal form (the control), and one receives the altered form (the variable). Once the test has run its course, you can compare the two versions and see which one received more dollars and donations.

One example of a good A/B test to try is comparing a plain pop-up recurring ask with a pop-up recurring ask that includes an added incentive for those who sign up, like a bumper sticker. We’ve found that this is one of the best ways to get recurring donations.

Setting up an a/b test

To set up an A/B test, go to your contribution form and then click on the “A/B Test” tab of your Form Tools menu. Give your test a descriptive name. The name will only be seen internally, so choose something that works best for you.

Settingup1

Underneath the name field, the “Automatic traffic allocation” option will automatically be selected (pictured below). This feature, also known as a multi-armed bandit algorithm, directs more traffic to the version of your form that is performing best so you won’t lose out on potential contributions, while still running a sound test. This is an easy, and highly recommended, way to run tests and raise funds at the same time. You can learn more about the multi-armed bandit on our blog.

If you uncheck the “Automatic traffic allocation” box, both of your forms will be allotted the same volume of traffic, even if one performs very poorly.

Select the feature that you would like to test:

  • Form title
  • Contribution form pitch
  • Branded layout
  • Fundraising thermometer
  • Embedding a video
  • Using a countdown
  • Pop-up recurring title
  • Pop-up recurring ask

This will be your variable. While you can test multiple features at once, if you want to run the most accurate test possible, you should pick just one. That way you will know it was the variation that caused a difference in visits, contributions, or contribution amounts. 

Settingup2

Once you’ve selected which feature you want to test, update the variation of that feature as needed. For example, if you are testing the form title, your current form title will populate and you’ll need to enter a second variation of the title (pictured below). 

Settingup3

Once you’ve finalized your test variations, click the blue “Create Test” button. Congrats — your test is live!

TRACKING YOUR A/B TEST

Once you create the test, you’ll be able to track each variations’ progress by returning to the “A/B Test” tab of the Form Tools menu for that form. This will show you traffic allocation, conversion rate, number of people who donated (conversions), number of visits, average contribution, and dollars per visit for each version of the form.
 

TrackingAB1
TrackingAB2

If you did NOT use the automatic traffic allocation (multi-armed bandit) option, you will need to click “Make Winner” on the variation that performed best in order to stop the less-successful form from being distributed. If you did use the multi-armed bandit method, the A/B test will do this on its own over time, but if you see a clear winner you can click “Make Winner” to speed up the process. 

Data on your A/B test will be included in your contributor CSV, which can be found in the “Statistics” tab of your form in the “Additional tools” dropdown menu at the bottom of the page. If you've offered any additional incentives, like a bumper sticker on a pop-up recurring test, information for all of the donors that signed up will be in the CSV as well.

If you have further questions about A/B tests or best practices, let us know.