# Session 4 Hypothesis Testing

## 4.1 Core content

- What is a good hypothesis?
- What is the relationship between hypothesis testing and causal inference? Why test hypotheses when we want to learn about the causal effect of some intervention on some outcome?
- What is a hypothesis test?
- What is a null hypothesis?
- What is a test statistic? When might we want to not use the difference of means as a test statistic? Estimators versus test statistics.
- Where does the reference distribution for a hypothesis test come from? In an experiment, it comes from the experimental design and the randomization.
- What is a \(p\)-value? How should we interpret the results of hypothesis tests?

- What do we want from a hypothesis test?
- A good test casts doubt on the truth rarely (i.e., has a controlled and low false positive rate)
- A good test easily distinguishes signal from noise (i.e., casts doubt on falsehoods often; has high statistical power)

- How would we know when our hypothesis test is doing a good job? (Power
analysis is its own session)
- What is a false positive rate? What is correct coverage of a confidence interval? (And why are we mentioning confidence intervals when we talk about hypothesis tests?)
- How might we assess the false positive rate of a hypothesis test for a given design and choice of test statistic? (The case of cluster-randomized trials and robust cluster standard errors.)

**Analyze as you randomize**in the context of testing for causal inference in a randomized experiment means that we think of the reference distribution that generates our \(p\)-value as arising from repetitions of the randomization process of the experiment.

## 4.2 Slides

Below are slides with the core content that we cover in this session.

Slides from previous Learning Days

Hypothesis Testing Lecture as used in Learning Days 10 Bogota