Session 5 Estimands and Estimators

5.1 Core content

Review: A causal effect, \(\tau_i\), is a comparison of unobserved potential outcomes for each unit \(i\): examples \(\tau_{i} = Y_{i}(Z_{i}=1) - Y_{i}(Z_{i}=0)\) or \(\tau_{i} = \frac{Y_{i}(Z_{i}=1)}{ Y_{i}(Z_{i}=0)}\).

  • To learn about \(\tau_{i}\), we can treat \(\tau_{i}\) as an estimand or target quantity to be estimated (today) or as a target quantity to be hypothesized about (recall session on hypothesis testing).

  • Many focus on the Average Treatment Effect (ATE), \(\bar{\tau}=\sum_{i=1}^n \tau_{i}\), in part, because it allows easy estimation

  • An estimator is a recipe for calculating a guess about the value of an estimand. For example, the difference of observed means for \(m\) treated units is one estimator of \(\bar{\tau}\): \(\hat{\bar{\tau}} = \sum_{i=1}^n (Z_i Y_i)/m - \sum_{i=1}^n ( ( 1 - Z_i) Y_i)/(n-m)\).

  • Different randomizations will produce different values of the same estimator targeting the same estimand. A standard error summarizes this variability in an estimator.

  • A \(100(1-\alpha)\)% confidence interval is a collection of hypotheses that cannot be rejected at the \(\alpha\) level. We tend to report confidence intervals containing hypotheses about values of our estimand and use our estimator as a test statistic.

  • Estimators should (1) avoid systematic error in their guessing of the estimand (be unbiased); (2) vary little in their guesses from experiment to experiment (be precise or efficient); and perhaps ideally (3) converge to the estimand as they use more and more information (be consistent).

  • Analyze as you randomize in the context of estimation means that (1) our standard errors should measure variability from randomization and (2) our estimators should target estimands defined in terms of potential outcomes.

  • We do not control for background covariates when we analyze data from randomized experiments. But covariates can make our estimation more precise. This is called covariance adjustment. Covariance adjustment in randomized experiments differs from controlling for in observational studis.

5.2 Slides

Below are slides with the core content that we cover in this session.

R Markdown Source

PDF Version

HTML Version

5.3 Resources

5.4 Quizzes and Exercises

5.5 Examples

References

Gerber, Alan S., and Donald P. Green. 2012. Field Experiments: Design, Analysis, and Interpretation. New York, NY: W. W. Norton & Company.