Research and Testing

Are hopeful email subject lines more effective than scary ones? Is our GOTV program effectively turning out voters? We test as one way to learn the answers to these and many more questions. We test because our assumptions and gut feelings are frequently wrong. “Testing” is the process of carefully trying out variations in what you do and measuring the results so you can learn what works. Download the most recent version of this slidedeck here.

Additional tools and resources to support your work:

Field Testing: A Climate Advocate’s Guide
Tipsheet: Field Testing: A Climate Advocate’s Guide
A step by step guide to the basics of testing
Experimentation in the time of coronavirus
Insight: Experimentation in the time of coronavirus
Applying an evidence-based framework to communications planning (Part 2 of our Fall 2020 communications series)
Cost effective survey methods
Webinar: Cost effective survey & testing methods
Online survey and message testing platforms can be inexpensive, quick, and easy to use. In this webinar, representatives from five vendors demonstrated their products.
Research question development worksheet
Worksheet: Research question development
Prompts to help you think through developing a research project, from big-picture strategy to the specifics of a hypothesis
 
Check out more content on this topic in our Resource Library!
Key Points
  • Step #1: Develop your question. Check what might already be known, then craft a clear question you want to answer via your test, and ideally a predicted answer of what will work and, most importantly, why. 
  • Step #2: Design. Lay out how the test will be conducted (such as how many different email subject lines you want to compare), and create those versions to be tested, carefully varying only that one element (e.g., subject line) in question. Decide how you will randomly determine who gets which version. 
  • Step #3: Collect Data. After planning (in detail) how your data will be organized, run your program with the variations (e.g., send email) and collect the results (such as rates of opens, clicks, actions). Keep an eye on the data throughout the test to ensure everything is going according to plan. 
  • Step #4: Analyze. Analysis means systematically examining your measurements to draw insights for your work – in other words, determining the “winning” version in your test.   
  • Step #5: Implement! Integrate your insights into your programs, keeping aware that insights from a test in one context might not apply for other people, places and times; it’s good to plan for this stage from the very beginning of the project.