Session 7 Threats to internal validity of randomized experiments

7.1 Core content

  • Missing data on the outcome (attrition) is especially a problem if the patterns of missingness are caused by the treatment itself.
    • Do not drop observations that are missing outcome data from your analysis.
    • You may be able to bound treatment effect estimates.
  • The effect of treatment assignment is not the same as the effect of receiving the treatment. Sometimes units will not comply with their assigned treatment status, but not all hope is lost.
    • One-sided compliance when some units assigned to treatment fail to take the treatment, but all units assigned to control do not take the treatment.
    • The “local average treatment effect” (LATE, also known as the “complier average causal effect,” CACE) is the average effect for the units who take the treatment when assigned, but not otherwise. With an exclusion restriction assumption, we can estimate the LATE from a randomized experiment.
  • “Spillover effects” or interference between units is a violation of one of the core assumptions for causal inference (Causal Inference).
    • However, this may not be a problem if you are interested in spillover effects and/or have designed your research to account for it.
  • Treating treatment and control units differently, such as with different data collection processes or extra attention to the treated units, can be problematic (Hawthorne effect).

7.2 Slides

Below are slides with the core content that we cover in our lecture on threats. You can directly use these slides or make your local copy and edit.

R Markdown Source

PDF version

html version

Slides from previous Learning Days

Lecture on Attrition and Missing Data as used in Learning Days 10 Bogota

Lecture on Threats as used in Learning Days 11 Benin

7.4 Quizzes and Exercises

  • The compliance dance. Randomly assign students to dance or not dance to music. We usually get two-sided non-compliance with this exercize, but it can be used to explain one-sided non-compliance.

7.5 Examples