Causal inference:
Vivalt, E. (2020). “How Much Can We Generalize from Impact Evaluations?”, Journal of the European Economics Association, 18(6): 3045–3089. Online Appendices, Presentation, Teaching Slides
Vivalt, E. (2019). “Specification Searching and Significance Inflation Across Time, Methods and Disciplines”, Oxford Bulletin of Economics and Statistics, 81(4): 797–816.
Vivalt, E. (2015). “Heterogeneous Treatment Effects in Impact Evaluation”, American Economic Review, Papers and Proceedings, 105(5): 467–470.
Media coverage of research on this theme:
- The Atlantic: “Make Science More Reliable, Win Cash Prizes”
- Vox: “Don’t teach a man to fish. Just give him the goddamn fish”
- The Washington Post: “The Wonkblog Guide to Holiday Giving”
- Mother Jones: “Most Studies of Social Interventions Are Pretty Worthless”
- Lant Pritchett’s mini-series for the Center for Global Development: “Is Your Impact Evaluation Asking Questions That Matter? A Four Part Smell Test”
- Marginal Revolution: “Everyone in development economics should read this paper”
- 3ie’s blog: “Trends in impact evaluation: Did we ever learn?”
Popular articles, podcasts, blog posts and talks:
- Harvard Business Review: “How to Be a Smart Consumer of Social Science Research”
- 80,000 Hours podcast: “Dr Eva Vivalt’s research suggests social science findings don’t generalize. So evidence-based development – what is it good for?”
- Centre for the Study of African Economies (CSAE) Conference 2018, Closing plenary: “Limits to evidence-based interventions for development”
- Effective Altruism Global: Global Poverty talk
- The Inter-American Development Bank’s blog: “How much do impact evaluations (really) help policymaking?”
- The NYU Development Research Institute’s blog: “5 ways to improve your impact evaluation”
- The World Bank’s Development Impact blog: “What do 600 papers on 20 types of interventions tell us about how much impact evaluations generalize?” and “What isn’t reported in impact evaluations?”
Cash transfer programs:
OpenResearch’s Basic Income RCT with Alex Bartik, David Broockman, Sarah Miller, and Elizabeth Rhodes (formerly Y Combinator Research)
This is a large project in the U.S. that will test the impact of receiving $1,000/month, unconditionally, for 3 years. Outcomes covered (some of these may merge but they are currently envisaged as separate papers):
- Employment, work quality, and job search
- Time use
- Income, expenditures, and financial health
- Mental and physical health outcomes
- Cognitive outcomes
- Material wellbeing
- Subjective, psychological, and social wellbeing
- Political and social attitudes
- Children’s outcomes
- Intrahousehold outcomes and intimate partner violence
- Migration and housing outcomes
- Crime
Bartik, A., Broockman, D., Miller, S., Rhodes, E., and Vivalt, E. (2021). “A Social Recession: Time Use during the COVID-19 Pandemic”
“The Unclaimed Property Puzzle: Billion Dollar Bills Lying on the Sidewalk” (sole PI)
Forecasting and development economics:
DellaVigna, S., Pope, D. and Vivalt, E. (2019). “Predict Science to Improve Science”, Science, 366(6464): 428-429.
DellaVigna, S., Otis, N. and Vivalt, E. (2020). “Forecasting the Results of Experiments: Piloting an Elicitation Strategy”, AEA Papers and Proceedings, 110(5): 75-79.
Social Science Prediction Platform website
The above papers are part of a larger project generously supported by the Alfred P. Sloan Foundation and an anonymous foundation.
Vivalt, E. and Coville, A. (2020). “How Do Policy-Makers Update Their Beliefs?”
Online appendix: Forecasting survey on the Social Science Prediction Platform (with .qsf file)
Vivalt, E., Coville, A. and KC, S. (2021). “Weighing the Evidence: Which Studies Count?”
Vivalt, E. and Coville, A. (2020). “Policy-Makers Consistently Overestimate Program Impacts”
Vivalt, E. (2020). “Using Priors in Experimental Design: How Much Are We Leaving on the Table?” in Bédécarrats F., I. Guérin, and F. Roubaud, eds., Randomized Control Trials in the Field of Development: The Gold Standard Revisited (pp. 293-303). London: Oxford University Press.
Chow, V. and Vivalt, E. (forthcoming). “Challenges in Changing Social Norms: Evidence from Interventions Targeting Child Marriage in Ethiopia.” Journal of African Economies.
Media coverage of research on this theme:
- Ars Technica: “‘I Could’ve Told You That’ Might Have a Useful Role to Play in Science”
- Science Daily: “Were those experiment results really so predictable? These researchers aim to find out”
- Behavioral Scientist: “Solving the Problem of ‘Obviousness’ with Prediction Platforms”
Popular articles, podcasts, blog posts and talks:
- Hear This Idea: “Evidence-Based Policy and Forecasting Social Science”
- Toronto Data Workshop: Keynote presentation
- The Conversation: “Predicting Research Results Can Mean Better Science and Better Advice”