Direct mail is both an art and a science.  And sometimes a mystery.

Have you ever had a package bomb that you thought would be a sure-fire winner? Who hasn’t?! Best practices don’t always hold up. What donors like may not be what they respond to. And while many people at your nonprofit are likely to have strong opinions, few people know with certainty what will or will not work in a direct mail campaign.

That can all sound a bit overwhelming. The good news is direct mail allows us to test! We can answer so many questions as to what works and does not work through testing, and we can build upon our findings to further strengthen results.

Here are the basic steps to running an effective test:

Step 1: State the goal of your mailing.

This might seem unnecessary, but a good starting point is to make sure everyone is on the same page as to the goal of your mailing. It could be to increase the number of new donors acquired, or to increase the average gift. Just putting it in writing will be helpful. 

Step 2: Come up with a hypothesis to test. 

You need to be able to clearly state what you are testing and the reasoning behind it. For example, your hypothesis could be that a more personal letter will lead to higher net income. 

Step 3: Design the test.

Design your test around your hypothesis. In the hypothesis example above, your test package might include the use of a handwritten font and variable data to personalize giving information. 

Step 4: Set up the test. 

Make sure the data is being pulled correctly, and that the test and control groups are large enough to yield statistically significant results. 

Step 5: Measure the results. 

Determine if the test or control won, and be sure to document your results. 

How often should I test?

The best nonprofits test relentlessly. 

Every housefile package and prospecting package should include a test. Don’t be satisfied with the status quo, even with a top performing appeal. You want to be continuously testing with the goal of improving results. 

Consider longer term tests as well. This could include tests such as decreasing the number of solicitations donors who give just once a year receive, or testing welcoming new donors to your organization using a different series of mailings and emails. Tests like these will take months or even a year to determine the winning treatment. 

Plan Ahead

Plan out your tests for the year, just like you plan your mailings for the year. Create a testing calendar for the year, and have a list of what you might want to test.

Testing should be part of your mailing strategy, and should be discussed early on. Don’t have it be a last-minute thing or feel like one more thing you have to do. 

Don’t let anyone use the excuse “we don’t have time to set up a test.” You are doing your nonprofit and donors a disservice when you don’t test. 

Keep track of the results

Your tests don’t matter if you don’t document your results. You’ll never remember what you learned! 

Set up a system where you are creating a report showing the test results, detailing which version won. You also want to create a summary document of all test results that will allow you to quickly answer the question “have we tested that before?”

Get testing inspiration from others

Look at what others are sending in the mail, both nonprofits and companies. That will help inspire ideas. If you see a package multiple times, it means it is working! Pay attention to those packages and think about what techniques you could test within your own program.

The best way to get on mailings lists is to make a donation to other nonprofits. This could be organizations similar to yours, or ones completely different. You can learn valuable insights from both. 

You will get lots of ideas! Of course, remember, that what works for another organization might not work for you. That’s why you will want to test.

Test boldly

Too often nonprofits test minor elements. When you test small changes, expect to see a small change in results. Usually these types of tests are insignificant, or only move the needle a tiny amount. It is often a waste of time.

If you want a big breakthrough, you need to be brave and test boldly! Test a completely different package, or redesign your control package. Think outside the box and be creative. Don’t be afraid to fail! Remember, you are just testing so the risk is relatively low, but the payout could be huge. 

One of the best parts of direct mail is the ability to continuously test, and learn. Be thoughtful about the testing process. You should plan ahead and be strategic about what, when and why you are testing. Ultimately, a successful testing program will lead to stronger results and happier donors.