Module 4 Randomization
The module on causal inference discussed the crucial role of randomization for drawing valid inferences from a comparison of treated and untreated groups. In this module, we move from theory to the first of many concrete choices for your research design.
We introduce four common ways to randomize treatment – simple, complete, block, and clustered – and when these different types of randomization may be available and appropriate. We also cover several popular designs including factorial designs and encouragement designs. The module provides some guidance on implementation, including best practices for checking for balance and ensuring replicability.
4.1 Core Content
What is randomization? Random assignment is not the same as random sampling.
Four common ways to randomize treatment:
Simple: randomly assign units to treatment (like a coin flip).
Complete: within a list of eligible units, a assign a fixed number to receive a treatment (like drawing from a urn).
Block (or stratified): assign treatment within specific strata or blocks, as if you are running an experiment within each block.
Cluster: assign groups (clusters) of observations to the same treatment condition.
Some popular designs:
Randomized access: randomization to availability of a treatment.
Randomized delayed access: randomize the timing of access.
Factorial: randomize units to combinations of treatment arms.
Encouragement: randomize the invitation to receive treatment.
How do you check whether your randomization produced balance on observables? Typically we conduct randomization tests also known as balance tests using the \(d^2\) omnibus test from
RItoolspackage (because it is randomization inference) or approximate this result with an \(F\)-test.
There are, of course, limits to randomization. We discuss some here and direct you to the module on threats for more.
Below are slides with the core content that we cover in our lecture on randomization. You can directly use these slides or make your own local copy and edit.
You can also see the slides used in previous EGAP Learning Days:
4.3.1 EGAP Methods Guides
EGAP Methods Guide 10 Things You Need to Know About Randomization
EGAP Methods Guide 10 Things You Need to Know About Cluster Randomization
4.3.2 Books, Chapters, and Articles
Standard operating procedures for Don Green’s lab at Columbia University. A comprehensive set of procedures and rules of thumb for conducting experimental studies.
Glennerster and Takavarasha, Running Randomized Evaluations. Chapter 2 on randomization.
Gerber and Green, Field Experiments. Chapter 2: Causal Inference and Experimentation
4.3.3 EGAP Policy Briefs
Randomizing delayed access
Blocked cluster randomization
- RItools, a set of tools for randomization-based inference including balance testing.