Module 6 Estimands and Estimators

6.1 Core content

Review: A causal effect, \(\tau_i\), is a comparison of unobserved potential outcomes for each unit \(i\). For example, this can be a difference or a ratio of unobserved potential outcomes.

  • To learn about \(\tau_{i}\), we can treat \(\tau_{i}\) as an estimand or target quantity to be estimated (this module) or as a target quantity to be hypothesized about (TODO: ADD LINK TO MODULE on hypothesis testing).

  • Many focus on the Average Treatment Effect (ATE), \(\bar{\tau}=\sum_{i=1}^n \tau_{i}\), in part, because it allows easy estimation

  • An estimator is a recipe for calculating a guess about the value of an estimand. For example, the difference of observed means for \(m\) treated units is one estimator of \(\bar{\tau}\).

  • Different randomizations will produce different values of the same estimator targeting the same estimand. A standard error summarizes this variability in an estimator.

  • A \(100(1-\alpha)\)% confidence interval is a collection of hypotheses that cannot be rejected at the \(\alpha\) level. We tend to report confidence intervals containing hypotheses about values of our estimand and use our estimator as a test statistic.

  • Estimators should (1) avoid systematic error in their guessing of the estimand (be unbiased); (2) vary little in their guesses from experiment to experiment (be precise or efficient); and perhaps ideally (3) converge to the estimand as they use more and more information (be consistent).

  • Analyze as you randomize in the context of estimation means that (1) our standard errors should measure variability from randomization and (2) our estimators should target estimands defined in terms of potential outcomes.

  • We do not control for background covariates when we analyze data from randomized experiments. But covariates can make our estimation more precise. This is called covariance adjustment. Covariance adjustment in randomized experiments differs from controlling for in observational studies.

TODO: Slides “At the moment we do not cover block-randomized, cluster-randomized, or binary outcomes or covariance adjustment. Meanwhile look at Chapter 3 of Gerber and Green and see the links and citations in the estimatr package for R”

6.2 Slides

Below are slides with the core content that we cover in this session.

R Markdown Source

PDF Version

HTML Version

6.3 Resources

6.3.2 Books and Chapters

  • Chapter 3 in Gerber and Green (2012)

6.3.3 EGAP Policy Briefs

6.3.4 Tools


Gerber, Alan S., and Donald P. Green. 2012. Field Experiments: Design, Analysis, and Interpretation. New York, NY: W. W. Norton & Company.