Probability levels and errors

A statistical decision that must be made at the beginning of your dissertation research involves statistical probability levels and potential errors. The conventional statistical levels for rejecting the null hypothesis are either .05 or .01. One (.01) is more statistically conservative than the other (.05) because with .01 you are less willing to have your results due to chance alone. You will accept only one time in one hundred that your statistical results were due to chance. A really conservative statistical probability level is .001.

1. A Type I Error occurs when you statistically reject the null and it was actually true. We conclude falsely that there was a statistical difference when there was none.

To make this type of error can be costly if you are deciding among alternatives. You are claiming that your new program or treatment is statistically better than the old one, but this is not true. If you are replacing the old with the new, your Type I error has resulted in lost dollars and other resources when the old one was just as good or better.

A statistical test with a large sample size can decrease the Type I error from occurring. However, a large sample size may also produce a significant difference that is not meaningful because the mean scores are only slightly different from each other.

Power is the probability of detecting an effect, given the effect is really there. You reject the null when it is actually false. Power of .80 means that 80% of the time you would find statistically significant differences. Power will be discussed more when sampling is addressed.

2. A Type II Error occurs when you statistically accept the null and it was in fact false. We conclude that there are no statistical differences when in fact there were!

This may not be as costly as a Type I error if the status quo is maintained due to your findings. However, if the new treatment or program you are proposing is better, then your results are maintaining a less desirable alternative.

If the hypothesis is true, you statistically accept the null.

If the hypothesis is false, you statistically reject the null.

If the hypothesis is true, and you reject the null, you commit a Type I error. (The statistical probability level is liberal.)

If the hypothesis is false, and you accept the null, you commit a Type II error. (The statistical probability level is conservative.)

Return from probability levels and errors to dissertation research questions and hypotheses.