CPH Focus: Evidence-Based Approaches to Public Health: Hypothesis Testing: Type I and Type II Errors

In this tutorial, we explore the fundamental concepts of Type I and Type II errors in hypothesis testing. These error types are critical to understanding the risk of false positives and false negatives within a research or applied epidemiology context. Recognizing these errors and their implications is vital for the Certified in Public Health (CPH) exam and for interpreting research findings accurately.

By the end of this tutorial, you will understand what Type I and Type II errors are, how they impact research, and how to minimize them. Practice questions with answers are included to help reinforce your understanding.

Table of Contents:

1. Introduction to Hypothesis Testing
2. Type I Error (False Positive)
   • Definition of Type I Error
   • Significance Level (α)
   • Consequences of Type I Error
3. Type II Error (False Negative)
   • Definition of Type II Error
   • Power of a Test
   • Consequences of Type II Error
4. Balancing Type I and Type II Errors
5. Practice Questions
6. Conclusion

1. Introduction to Hypothesis Testing

Hypothesis testing is a statistical method used to make decisions about a population based on sample data. The process involves formulating two hypotheses:

• Null hypothesis (H₀): Assumes that there is no effect or no difference.
• Alternative hypothesis (H_A): Assumes that there is an effect or a difference.

The goal of hypothesis testing is to determine whether there is enough evidence in the sample data to reject the null hypothesis in favor of the alternative hypothesis. However, this process is subject to two types of errors: Type I error and Type II error.

2. Type I Error (False Positive)

A Type I error, also known as a false positive, occurs when the null hypothesis is incorrectly rejected when it is actually true. In other words, a Type I error happens when we conclude that there is an effect or difference when, in reality, there is none.

2.1 Definition of Type I Error

A Type I error occurs when the data lead us to reject the null hypothesis (H₀) even though it is true. This means that the conclusion that an effect exists is incorrect.

2.2 Significance Level (α)

The probability of making a Type I error is denoted by α, which is known as the significance level. Common significance levels are 0.05 (5%) or 0.01 (1%), meaning there is a 5% or 1% chance of incorrectly rejecting the null hypothesis.

• α = 0.05: There is a 5% chance of committing a Type I error (false positive).
• α = 0.01: There is a 1% chance of committing a Type I error, which is more conservative and reduces the risk of a false positive.

2.3 Consequences of Type I Error

In public health research, a Type I error can have serious consequences. For example, concluding that a new treatment is effective when it is not could lead to widespread use of an ineffective or harmful intervention. Therefore, controlling the significance level is crucial to minimize the risk of a Type I error.

3. Type II Error (False Negative)

A Type II error, also known as a false negative, occurs when the null hypothesis is not rejected when it is actually false. This means that the test fails to detect an effect or difference that truly exists.

3.1 Definition of Type II Error

A Type II error occurs when we fail to reject the null hypothesis (H₀) even though the alternative hypothesis (H_A) is true. This means that the conclusion that there is no effect is incorrect because an effect does exist.

3.2 Power of a Test

The probability of making a Type II error is denoted by β. The power of a statistical test is the probability of correctly rejecting the null hypothesis when the alternative hypothesis is true, and it is given by 1 – β.

• Power: The ability of a test to detect an effect when there is one. A higher power reduces the likelihood of a Type II error.
• β (Type II error rate): The probability of failing to detect an effect when there is one. Lowering β reduces the risk of a false negative.

Increasing the sample size or using more precise measurements can increase the power of a test, thereby reducing the likelihood of a Type II error.

3.3 Consequences of Type II Error

In public health research, a Type II error can be problematic because it means that a true effect goes undetected. For example, failing to identify an effective treatment could result in missed opportunities for improving health outcomes. Therefore, minimizing the probability of a Type II error is important to avoid overlooking meaningful findings.

4. Balancing Type I and Type II Errors

There is a trade-off between Type I and Type II errors. Reducing the risk of a Type I error (by lowering α) increases the risk of a Type II error, and vice versa. This balance must be carefully managed based on the context of the research.

• Lowering α: Reduces the chance of a false positive but increases the chance of a false negative.
• Increasing power: Reduces the chance of a Type II error but may require a larger sample size or more resources.

In public health research, the cost of making one type of error may be higher than the other, depending on the situation. For example, in the case of a new drug, it may be more important to avoid a Type I error (false positive) to prevent the approval of an ineffective or harmful drug.

5. Practice Questions

Test your understanding of Type I and Type II errors with these practice questions. Try answering them before checking the solutions.

Question 1:

A researcher concludes that a new public health intervention is effective when it actually is not. What type of error is this?

Answer 1:

Question 2:

A study fails to detect a significant effect of a new treatment, even though the treatment is actually effective. What type of error is this?

Answer 2:

Question 3:

If the significance level (α) is set to 0.01 instead of 0.05, what happens to the risk of Type I and Type II errors?

Answer 3:

6. Conclusion

Understanding Type I and Type II errors is essential for making accurate decisions in public health research. Researchers must carefully balance the risk of false positives and false negatives by setting appropriate significance levels and ensuring studies have sufficient power. This balance helps ensure that research findings are both reliable and actionable.

Key Takeaways:

• Type I error: Occurs when the null hypothesis is incorrectly rejected, leading to a false positive. The probability of this error is denoted by α.
• Type II error: Occurs when the null hypothesis is not rejected even though it is false, leading to a false negative. The probability of this error is denoted by β, and the power of the test (1 – β) reflects the test’s ability to detect a true effect.
• Balancing errors: Reducing the risk of one type of error often increases the risk of the other, so researchers must carefully choose significance levels and power based on the context of the study.

Always be aware of the consequences of both types of errors, and ensure that your study design reduces their impact as much as possible to avoid the pitfalls in evidence-based public health scenarios.

Humanities Moment

The featured image for this CPH Focus is Interior with Two Figures, Open Window (1922) by Henri Matisse (French, 1869 – 1954). Mattise was a visionary artist whose innovative use of color and expressive line influenced the course of modern art in the twentieth century. Renowned for his painting, drawing, sculpture, and later his striking cut-paper collages, Matisse helped define movements such as Fauvism and continued evolving his style throughout his life. His ability to balance rigorous structure with decorative beauty secured his legacy as one of the most important artists of his era.