A/B Testing

A/B testing is the practice of randomly showing one version of a design (A) to half of your users and a modified version (B) to the other half. Responses are recorded and measured for statistical analysis.

A/B testing is a terrific way to validate opinions, resolve differing opinions, and make incremental changes to your product. It eliminates unnecessary risk and guesswork from the design process and is inexpensive and easy to make informed decisions.

A/B Testing Examples

A/B testing examines any change to your product that affects the user experience. These changes include but are not limited to:

• Button color, placement, shape, or size
• Font family, style, or weight
• Illustration style
• Photography style
• Form label or placement
• Text, tone, or voice

Conducting A/B Testing

A/B testing is an experiment with several vital steps.

1. Identify a goal, clearly articulating what you hope to learn with your A/B test.
2. Form a hypothesis, being sure to identify a measurable outcome.
3. Pick only one variable to change.
4. Determine your test sample size based on how many people use your product and how many visitors you need for results to be statistically significant.
5. Determine the length of your experiment. Depending on the sample size, a test can run for one week or up to several months.
6. Measure the statistical significance of your results. Were the differences a result of chance or directly related to the variable?
7. Implement the results of your test and plan for the next one.