Statistics
Textbooks
Boundless Statistics
A Closer Look at Tests of Significance
A Closer Look at Tests of Significance
Statistics Textbooks Boundless Statistics A Closer Look at Tests of Significance A Closer Look at Tests of Significance
Statistics Textbooks Boundless Statistics A Closer Look at Tests of Significance
Statistics Textbooks Boundless Statistics
Statistics Textbooks
Statistics
Concept Version 6
Created by Boundless

Was the Result Important?

The results are deemed important if they change the effects of an event.

Learning Objective

  • Distinguish the difference between the terms 'significance' and 'importance' in statistical assessments


Key Points

    • When used in statistics, the word significant does not mean important or meaningful, as it does in everyday speech; with sufficient data, a statistically significant result may be very small in magnitude.
    • Importance is a measure of the effects of the event. A difference can be significant, but not important.
    • It is preferable for researchers to not look solely at significance, but to examine effect-size statistics, which describe how large the effect is and the uncertainty around that estimate, so that the practical importance of the effect may be gauged by the reader.

Terms

  • null hypothesis

    A hypothesis set up to be refuted in order to support an alternative hypothesis; presumed true until statistical evidence in the form of a hypothesis test indicates otherwise.

  • statistical significance

    A measure of how unlikely it is that a result has occurred by chance.


Full Text

Significance vs. Importance

Statistical significance is a statistical assessment of whether observations reflect a pattern rather than just chance. When used in statistics, the word significant does not mean important or meaningful, as it does in everyday speech; with sufficient data, a statistically significant result may be very small in magnitude.

If a test of significance gives a $p$-value lower than or equal to the significance level, the null hypothesis is rejected at that level . Such results are informally referred to as 'statistically significant (at the $p=0.05$ level, etc.)'. For example, if someone argues that "there's only one chance in a thousand this could have happened by coincidence", a $0.001$ level of statistical significance is being stated. Once again, this does not mean that the findings are important.

$p$-Values

A graphical depiction of the meaning of $p$-values.

So what is importance? Importance is a measure of the effects of the event. For example, we could measure two different one-cup measuring cups enough times to find that their volumes are statistically different at a significance level of $0.001$. But is this difference important? Would this slight difference make a difference in the cookies you're trying to bake? No. The difference in this case is statistically significant at a certain level, but not important.

Researchers focusing solely on whether individual test results are significant or not may miss important response patterns which individually fall under the threshold set for tests of significance. Therefore along with tests of significance, it is preferable to examine effect-size statistics, which describe how large the effect is and the uncertainty around that estimate, so that the practical importance of the effect may be gauged by the reader.

[ edit ]
Edit this content
Prev Concept
Data Snooping: Testing Hypotheses Once You've Seen the Data
The Role of the Model
Next Concept
Subjects
  • Accounting
  • Algebra
  • Art History
  • Biology
  • Business
  • Calculus
  • Chemistry
  • Communications
  • Economics
  • Finance
  • Management
  • Marketing
  • Microbiology
  • Physics
  • Physiology
  • Political Science
  • Psychology
  • Sociology
  • Statistics
  • U.S. History
  • World History
  • Writing

Except where noted, content and user contributions on this site are licensed under CC BY-SA 4.0 with attribution required.