# Session 4 Hypothesis Testing

## 4.1 Core content

• What is a good hypothesis?
• What is the relationship between hypothesis testing and causal inference? Why test hypotheses when we want to learn about the causal effect of some intervention on some outcome?
• What is a hypothesis test?
• What is a null hypothesis?
• What is a test statistic? When might we want to not use the difference of means as a test statistic? Estimators versus test statistics.
• Where does the reference distribution for a hypothesis test come from? In an experiment, it comes from the experimental design and the randomization.
• What is a $$p$$-value? How should we interpret the results of hypothesis tests?
• What do we want from a hypothesis test?
• A good test casts doubt on the truth rarely (i.e., has a controlled and low false positive rate)
• A good test easily distinguishes signal from noise (i.e., casts doubt on falsehoods often; has high statistical power)
• How would we know when our hypothesis test is doing a good job? (Power analysis is its own session)
• What is a false positive rate? What is correct coverage of a confidence interval? (And why are we mentioning confidence intervals when we talk about hypothesis tests?)
• How might we assess the false positive rate of a hypothesis test for a given design and choice of test statistic? (The case of cluster-randomized trials and robust cluster standard errors.)
• Analyze as you randomize in the context of testing for causal inference in a randomized experiment means that we think of the reference distribution that generates our $$p$$-value as arising from repetitions of the randomization process of the experiment.

## 4.2 Slides

Below are slides with the core content that we cover in this session.

R Markdown Source

HTML Version

Slides from previous Learning Days

Hypothesis Testing Lecture as used in Learning Days 10 Bogota

## 4.4 Quizzes and Exercises

Exercises on hypothesis testing

Exercise on hypothesis testing using R