Module 4 Randomization

The module on causal inference discussed the crucial role of randomization for drawing valid inferences from a comparison of treated and untreated groups. In this module, we move from theory to the first of many concrete choices for your research design.

We introduce four common ways to randomize treatment – simple, complete, block, and clustered – and when these different types of randomization may be available and appropriate. We also cover several popular designs including factorial designs and encouragement designs. The module provides some guidance on implementation, including best practices for checking for balance and ensuring replicability.

4.1 Core Content

• What is randomization? Random assignment is not the same as random sampling.

• Four common ways to randomize treatment:

  • Simple: randomly assign units to treatment (like a coin flip).

  • Complete: within a list of eligible units, a assign a fixed number to receive a treatment (like drawing from a urn).

  • Block (or stratified): assign treatment within specific strata or blocks, as if you are running an experiment within each block.

  • Cluster: assign groups (clusters) of observations to the same treatment condition.

• Some popular designs:

  • Randomized access: randomization to availability of a treatment.

  • Randomized delayed access: randomize the timing of access.

  • Factorial: randomize units to combinations of treatment arms.

  • Encouragement: randomize the invitation to receive treatment.

• How do you check whether your randomization produced balance on observables? Typically we conduct randomization tests also known as balance tests using the \(d^2\) omnibus test from xBalance in the RItools package (because it is randomization inference) or approximate this result with an \(F\)-test.

• There are, of course, limits to randomization. We discuss some here and direct you to the module on threats for more.

4.2 Slides

Below are slides with the core content that we cover in our lecture on randomization. You can directly use these slides or make your own local copy and edit.

• R Markdown Source

• PDF version

• HTML version

The linked files shows how to do replicable randomization in R. You can also see more examples of randomization in R at 10 Things You Need to Know About Randomization.

You can also see the slides used in previous EGAP Learning Days:

• The design issues presentation from EGAP Learning Days at the African School of Economics, Abomey-Calavi, June 2019 (first section reviews randomization designs)

• The randomization presentation from EGAP Learning Days at Universidad de Los Andes, Bogotá, April 2019

• The randomization presentation from EGAP Learning Days at Universidad Católica del Uruguay, Montevideo, March 2018

• The randomization presentation from EGAP Learning Days in Guatemala City, Guatemala, August 2017

• The randomization presentation from EGAP Learning Days in Salima, Malawi, February 2017

• The randomization presentation from EGAP Learning Days at Universidad Diego Portales in Santiago, Chile, May 2016

4.3 Resources

4.3.1 EGAP Methods Guides

• EGAP Methods Guide 10 Things You Need to Know About Randomization

• EGAP Methods Guide 10 Things You Need to Know About Cluster Randomization

4.3.2 Books, Chapters, and Articles

• Standard operating procedures for Don Green’s lab at Columbia University. A comprehensive set of procedures and rules of thumb for conducting experimental studies.

• Glennerster and Takavarasha, Running Randomized Evaluations. Chapter 2 on randomization.

• Gerber and Green, Field Experiments. Chapter 2: Causal Inference and Experimentation

4.3.3 EGAP Policy Briefs

Factorial designs

• EGAP Policy Brief 57: How Media Influence Social Norms: Evidence from Mexico

• EGAP Policy Brief 58: Does Bottom-Up Accountability Work?

Randomizing access

• EGAP Policy Brief 24: Reducing Elite Capture in the Solomon Islands

Randomizing delayed access

• EGAP Policy Brief 35: Reducing Reconvictions Among Released Prisoners

• EGAP Policy Brief 60: Reducing Youth Support for Violence through Training and Cash Transfers in Afghanistan

Cluster randomization

• EGAP Policy Brief 22: Getting Out the Vote

Blocked cluster randomization

• EGAP Policy Brief 54: Incumbent Malfeasance Revelations

• EGAP Policy Brief 56: Reporting Corruption

4.3.4 Tools

• RItools, a set of tools for randomization-based inference including balance testing.