In an evaluation, statistical significance refers to the probability that any differences found between the treatment group and control group are not due to chance but are the result of the treatment group’s participation in the program or intervention being studied. For example, if an outcome evaluation finds that after participating in a substance abuse program, the treatment group was statistically significantly less likely to abuse substances compared with the control group, this means that the difference between the two groups is likely due to the program and not due to chance.
In social science, researchers generally use a p-value of 0.05 or less, which means the probability that the difference between the treatment group and control group is due to chance is less than 5 percent. The p=0.05 is the cut-off point that CrimeSolutions.gov Expert Reviewers use to score whether an outcome is statistically significant. If the p-value is larger than 0.05, the outcome is not statistically significant, and the difference between the treatment and control group could be due to chance. See Program Review and Rating from Start to Finish for more information.