What does statistical significance mean and why should you care?

What is statistical significance?

Simply put, statistical significance tells you that there is a true and meaningful difference in the results of the data that you are measuring. Therefore, your results are real and did not occur just by chance. Take, for example, an email A/B test. If you’re not familiar with A/B testing, click here to read last week’s post. From the results, it appears that one email had a higher click-through rate. However, the difference in the click-through rates between the two emails may not have a statistically significant difference. Therefore, by testing the statistical significance in your data, you can be more confident in your decisions.

Why is statistical significance important?

You may be sitting there, wondering, what does statistical significance have to do with running a boutique fitness studio? Well, any business owner should understand this concept because it will help you make better and more informed decisions regarding your marketing strategies.

Let’s go back to our example of an email A/B test. As a result of the email campaign, you find that version A of your email has a higher click-through rate than version B. Therefore, you would assume that you should then go with version A of your email. However, after running your analysis to find out the statistical significance, you find that the difference in your results is not significant. Since your results are not statistically significant, it doesn’t matter which version of your email you chose to go forward with because the difference in your results is likely due to some external factor and not due to the differences in your ads.

If you hadn’t run this analysis on the resulting data from your campaign, you would have been led to believe version A was the better choice; however, now you know there is no “better choice.” On the other hand, imagine you did find out that the difference between the results is statistically significant. If that is the case, you could be confident in your decision to move forward with version A of your email. The point is that knowing this information will allow you to be more confident in making better marketing decisions for your business.

Tools:

Now that you understand how vital statistical significance is, you should start implementing it! And, it’s very straightforward. There are a wealth of online tools available that will allow you to calculate statistical significance seamlessly.

Check out this site to for a free tool to calculate statistical significance on your own. Or, if you want to learn more about online platforms that can make A/B testing and the data analysis more seamless here.

Miss last week’s post? Read about A/B testing.