Research Bite: Common Statistical Tests, Null Hypothesis, and Statistical Significance.

Statistics

In the simplest terms, statistics is the calculated differences between groups. In the case of a research study, in the simplest terms, statistics gives you information on the strength of the differences between groups with whatever dependent variable the researcher is observing. These groups can be the intervention group and control group or different intervention groups.

Null Hypothesis

In research, the focus is looking to disprove the null hypothesis and you use the results of your test statistic to do so. The null hypothesis notes that there is no relationship between the variables. Basically, this definition is stating that the differences between the groups (intervention and control, different intervention groups, etc.) has nothing to do with the addition of an independent variable. The differences seen between the groups are due to chance and not because of the independent variable.  

Null hypothesis, alpha, p-values, and statistical significance

Disproving the null hypothesis is done using a statistical test that is chosen based on characteristics of the groups being compared and distribution of the data.

The test statistic’s ability to determine the differences between the groups is based on the p value of that test statistic and the alpha level set by the researcher (usually .05 or lower). If the p-value is less than or equal to the alpha (p< .05), then the null hypothesis is rejected, the result is a statistically significant difference between the groups. If the p-value is greater than alpha (p > .05), then the null hypothesis is not rejected, the differences between the groups is statistically nonsignificant (n.s.).

Using percentages, if you set your alpha to .05 and the p value is less than or equal to .05, you are basically stating with 95% certainty that the differences you see between the groups is due to the intervention and not chance. If you set alpha lower (ex: .01 and the p value meets that alpha value), then you are basically stating with 99% certainty that the differences you see between the groups is due to the intervention and not chance.

A statistical test is chosen based on what types of groups you are comparing and the distribution of the data.  

Common statistical tests and when they are used

Parametric tests-used when the data is normally distributed.

• Pearson Correlation (r)-measures the statistical relationship between two variables. The Pearson correlation provides information on the direction and strength of the relationship, r has a maximum value of 1 and can be negative or positive. A positive r indicates a positive relationship between the variables, as one variable increases, the other variable increases as well. A negative r indicated a negative relationship between the variables, as one variable increases, the other variable decreases.
• T-test— Used when the mean and standard deviation of the population are known. T-tests are used when comparing two groups.
  1. Paired T-Test-Tests for the difference between two variables from the same population (pre- and post-test score).
  2. Independent T-test- determines whether there is a statistically significant difference between the means in two unrelated groups. For example -comparing boys and girls in a population.
  3. One sample t-test– The mean of a single group is compared with a given mean.

• Z-test– used when the variances are known, and the sample size is large

• ANOVA–used to determine if the means of two or more groups are significantly different from each other. ANOVAs are more accurate than t-tests when comparing two groups.

Non-parametric statistical tests-used when data is not normally distributed.

• Chi-square test (χ2 test)- chi-square test is used to compare two categorical variables.

• Mann-Whitney U Test compares the difference between two variables from the same population (pre- and post-test score) that does not follow a normal distribution.

Why does this matter?

When reading a research article, researchers will provide to you what statistical test they used to determine the difference between the groups to answer the research questions that are stated at the end of the literature review. Things to look out for, the research questions they are seeking to answer, the type of test statistic they use to answer those questions, the level at which they set their alpha/p-value, and the sample size.