Type I Error

In a test of significance, Type I error is the error of rejecting the null hypothesis when it is true — of saying an effect or event is statistically significant when it is not. The projected probability of committing type I error is called the level of significance. For example, for a test comparing two samples, a 5% level of significance (a = .05) means that when the null hypothesis is true (i.e. the two samples are part of the same population), you believe that your test will conclude “there’s a significant difference between the samples” 5% of the time.