The Theory and Practice of Field Experiments:
An Introduction from the EGAP Learning Days
An Introduction from the EGAP Learning Days
August 22, 2024
Module 1 Introduction
Over the past decade, Evidence in Governance and Politics (EGAP) has organized Learning Days workshops with the aim of building experimental social-science research capacity among principal investigators (PIs) – both researchers and practitioners – in Africa and Latin America. By sharing the practical and statistical methods of randomized field experiments with workshop participants, the Learning Days effort hopes to identify and nurture researcher networks around the world and to create strong, productive connections between these researchers and EGAP members.
The Learning Days workshops are a combination of design clinics, research presentations, guided work with statistical software, and topical lectures by a small group of instructors, largely professors and PhD students from the EGAP network. The workshops focus on methods for the design and analysis of randomized experiments in the field rather than on randomized experiments in the lab or non-randomized studies.
This book grew out of a desire to share the materials we developed for the Learning Days. The current version is written primarily for instructors and organizers of similar workshops and courses aimed at PIs — i.e., professors, post-doctoral fellows, PhD students, and NGO/government agency evaluators — who will implement randomized studies of programs related to institutions, governance, and development. Much of the material will also be useful as a refresher for past participants of the Learning Days workshops.
This book is a comprehensive overview of causal inference methods for researchers developing an experimental research design. It is organized in modules and covers topics such as causal inference, randomization, hypothesis testing, estimands, estimators, statistical power, measurement, threats to internal validity, and the ethics of experimentation. The modules appear in the order the Learning Days instructors have found most useful. However, the modules are linked to one another and can be reordered to suit your needs as an instructor. In the appendix, we include some course preliminaries including a glossary of terms and an introduction to R and RStudio.
The book includes slides on the core content, the EGAP Research Design Form, and references to research examples and slides used in previous Learning Days. This material builds significantly on and links to EGAP’s work on methodology, summarized in the EGAP Methods Guides. We have made significant extensions to the past Learning Days’ materials on hypothesis testing, estimation, and statistical power and added new modules on the research design process, measurement, and ethical considerations. The slides and modules presented here contain too much information to be covered in a single week, the usual length of a Learning Days workshop. We have chosen to present more rather than less information, however, to help instructors tailor their courses to their specific audiences.
1.1 How to use the book
To gain the most benefit from this book, please have R and RStudio installed on your machine. In fact, the slides assume that you will use Rmarkdown to adapt them for your own purposes.
To get going with R, see the module Introduction to R and RStudio.
You can copy this book or parts thereof (e.g., slides, etc.) either by using the Download button on the front page of http://github.com/egap/theory_and_practice_of_field_experiments or by using github directly (by forking this repository).
We are happy for anyone to use the materials as long as EGAP is attributed. See Creative Commons Attribution-ShareAlike 4.0 International License for the precise terms.
1.2 We would love to hear from you!
If you have any questions, feedback or have organized your own event, please get in touch! Simply post an issue on Github or make comments using hypothes.is in your browser and let us know via email, admin@egap.org. We’ll periodically go through the comments.
1.3 Acknowledgments
The materials included in this book have been developed over the past years by various Learning Days instructors. These include (in alphabetical order) Jake Bowers, Jasper Cooper, Ana De la O, Lindsay Dolan, Natalia Garbiras Díaz, Macartan Humphreys, Nahomi Ichino, Salif Jaiteh, Gareth Nellis, Dan Nielson, Rafael Piñeiro, Fernando Rosenblatt, Tara Slough, Peter van der Windt and Maarten Voors. We thank Natalia Garbiras Díaz, Macartan Humphreys, Anghella Brigeth Rosero Rodriguez, and Tara Slough in particular for comments on an early draft of the book. We also thank Brice Bado, Simon Chauchard, Jasper Cooper, Simone Dietrich, Thad Dunning, Jessica Gottlieb, Macartan Humphreys, Julien Labonne, Ambaliou Olounlade, Daniel Rubenson, and Saloua Zerhouni for their help in reviewing the French translation and Rosario Aguilar, Ana De la O, Pablo Egaña del Sol, Omar Garcia Ponce, Paul Lagunes, Luis Maldonado, Fernando Martel Garcia, Paula Muñoz, Raul Pacheco-Vega, Rafael Piñiero, Pablo Querubin, Mauricio Romero, Fernando Rosenblatt, Santiago Saavedra, Lucía Tiscornia, Santiago Tobón, and Juan Vargas for their help in reviewing the Spanish translation of these materials.
At EGAP, Matt Lisiecki, Ingrid Lee, Goldie Negelev, Max Mendez-Back and others have provided wonderful support. Learning Days have been generously funded by the Hewlett Foundation and supported by institutions around the world including the African School of Economics (Benin), Universidad Diego Portales (Chile), Universidad de los Andes (Colombia), Ghana Center for Democratic Development (Ghana), Mercy Corps (Guatemala), Invest in Knowledge (Malawi), NYU Abu Dhabi (UAE), Universidad Católica del Uruguay (Uruguay).
Author order randomized. https://jakebowers.org↩︎