A result is statistically significant when its p-value falls below the chosen alpha, commonly 0.05, meaning a true null hypothesis would produce data this extreme less than 5 percent of the time. Alpha is also the Type I error rate, the risk of declaring an effect that is not there. Engineers separate statistical significance from practical significance: a difference can be real and still too small to matter against the tolerance.
It's significant, sure, p of 0.001. But it's 0.2 percent of the tolerance. Nobody's changing a recipe for that.
Write 'statistically significant' in reports. Plain 'significant' gets read as 'big' by managers.
Also heard as
- statistically significant
- stat sig
Related terms
p-value
Probability of seeing a result at least this extreme if there were truly no effect, used to judge statistical significance.
Power
Probability that a test will detect a real effect of a given size, set by sample size, noise and alpha.
Confidence interval
Range calculated from sample data that brackets an unknown parameter, such as a mean, at a stated confidence level.
t-test
Test that compares a mean against a target, or two means against each other, using the t distribution.