Field Experiments
Kyiv School of Economics, April 2009
This is a 10-hour course on one of the most exciting developments in economics over the past 10 years: the use of field experiments. A crucial problem in empirical research is how to find data with exogenous variation, in order to be able to draw correct causal inferences. Natural experiments are one familiar source of clean data, but they only allow us to examine a limited number of topics, where we are lucky enough to see naturally occurring variation of the sort we need. Field experiments expand the range of topics we can examine; in this technique, the economist actually generates the data by running a controlled experiment.
In this course, we will study a variety of field experiments that have been performed in different areas of economics, including IO, labor, public finance, health, and development. Economists have been realizing that there are many opportunities to gain answers to economic questions by conducting field experiments, either on their own or in collaboration with firms, governments, or NGOs. My current work at Yahoo! Research focuses on the value of field experiments to firms.
The learning objectives in this course are (1) to add field experiments to your toolkit as an empirical economist, (2) to understand the relationships between field experiments, laboratory experiments, and natural experiments, (3) to develop a deeper understanding of the problem of causal inference in empirical economics, and (4) to learn valuable principles of experimental design. We will achieve these objectives by reading a variety of examples of field experiments in different areas of economics.
Each student will write a 5-10 page paper proposing a field experiment to be done as a research project in his or her area of interest. The due date for this paper will be Friday, May 8, 2009.
Students should read the required readings (marked with asterisks) before coming to class, and be prepared to participate in class discussions. I also encourage you to choose to give a 30-minute presentation to the class on one of the required papers in the course. If you do, this presentation will count for half of your course grade. The rest of your grade (50% for those who make a presentation, 100% for those who do not) will depend on the final paper.
Session 1: Introduction and Auctions - 9 April
*Lucking-Reiley, David. "Using Field Experiments to Test Equivalence Between Auction Formats: Magic on the Internet." American Economic Review, December 1999, vol. 89, no. 5, pp.1063-1080.
*List, John A., and David Lucking-Reiley. "Demand Reduction in Multi-Unit Auctions: Evidence from a Sportscard Field Experiment." American Economic Review, September 2000, vol. 90, no. 4, pp. 961-972.
*Angrist, Joshua D. and Alan B. Krueger. "Instrumental Variables and the Search for Identification: From Supply and Demand to Natural Experiments." Journal of Economic Perspectives, Fall 2001, vol. 15, no. 4, pp. 69-85.
Levin, Dan. "Demand Reduction in Multiunit Auctions: Comment." American Economic Review, March 2005, vol. 95, no.1, pp. 467-471.
Engelbrecht-Wiggans, Richard, John A. List, and David H. Reiley. "Demand Reduction in Multiunit Auctions: Reply." American Economic Review, March 2005, vol. 95, no.1, pp. 472-476.
Katkar, Rama, and David H. Reiley. "Public Versus Secret Reserve Prices in eBay Auctions: Results from a Pokemon Field Experiment." Advances in Economic Analysis and Policy, 2006, Volume 6, Issue 2, Article 7.
Reiley, David H. "Field Experiments on the Effects of Reserve Prices in Auctions: More Magic on the Internet." RAND Journal of Economics, Spring 2006, vol. 37, no. 1, pp. 195-211.
Resnick, Paul, Richard Zeckhauser, John Swanson, and Kate Lockwood. "The Value of Reputation on eBay: A Controlled Experiment." Experimental Economics, June 2006, vol. 9, no. 2, pp. 79-101.
Harrison, Glenn W., and John A. List. "Field Experiments." Journal of Economic Literature, December 2004, vol. 42, no. 4, pp. 1009-1055.
Levitt, Steven D., and John A. List. "Field Experiments in Economics: The Past, the Present, and the Future." NBER Working Paper 14356, September 2008.
Session 2: Charitable Fundraising (and Public Finance) - 10 April
*List, John A., and David Lucking-Reiley. "The Effects of Seed Money and Refunds on Charitable Giving: Experimental Evidence from a University Capital Campaign." Journal of Political Economy, February 2002, vol. 110, no. 8, pp. 215-233.
*Karlan, Dean, and John A. List. "Does Price Matter in Charitable Giving? Evidence from a Large-Scale Field Experiment." American Economic Review, December 2007, vol. 97, no. 5, pp. 1774-1793.
Breman, Anna. "Precommitments: Two Field Experiments on Inter-temporal Preferences in Charitable Giving." Working paper, University of Arizona, June 2008.
Falk, Armin. "Gift Exchange in the Field." Econometrica, September 2007, vol. 75, no. 5 , 1501–1511.
Frey, Bruno S, and Meier, Stephan. "Social Comparisons and Pro-social Behavior: Testing 'Conditional Cooperation' in a Field Experiment." American Economic Review, December 2004, vol. 94, no. 5, pp. 1717-1722.
Shang, Jen, and Rachel Croson. "Field Experiments in Charitable Contribution: The Impact of Social Influence on the Voluntary Provision of Public Goods." Working paper, University of Texas at Dallas, 2006. Forthcoming, Economic Journal.
Soetevent, Adrian. "Anonymity in Giving in a Natural Context - A Field Experiment in 30 Churches." Journal of Public Economics, December 2005, vol. 89, no. 11-12, pp. 2301-2323.
Cohen, Jessica, and Pascaline Dupas. "Free Distribution or Cost-Sharing? Evidence from a Randomized Malaria Prevention Experiment." Working paper, UCLA, March 2009.
Duflo, Esther, and Emmanuel Saez. "The Role of Information and Social Interactions in Retirement Plan Decisions: Evidence From a Randomized Experiment." Quarterly Journal of Economics, August 2003, vol. 18, no. 3, pp. 815-842.
Duflo, Esther, William Gale, Jeffrey Liebman, Peter Orszag, and Emmanuel Saez. "Saving Incentives for Low- and Middle-Income Families: Evidence from a Field Experiment with H&R Block." Quarterly Journal of Economics, November 2006, vol. 121, no. 4, pp. 1311-1346.
Kling, Jeffrey. "Methodological Frontiers of Public Finance Field Experiments." National Tax Journal, March 2007, vol. 60, pp. 109-127.
Session 3: Labor - 14 April, morning
*Bandiera, Oriana, Iwan Barankay, and Imran Rasul. "Social Preferences and the Response to Incentives: Evidence from Personnel Data." Quarterly Journal of Economics, August 2005, vol. 120, no. 3, pp. 917-962.
*Bertrand, Marianne, and Sendhil Mullainathan. "Are Emily and Greg More Employable than Lakeesha and Jamal? A Field Experiment on Labor-Market Discrimination." American Economic Review, September 2004, vol. 94, no. 4, pp. 991-1013.
Bandiera, Oriana, Iwan Barankay, and Imran Rasul. "Incentives for Managers and Inequality Among Workers: Evidence From a Firm-Level Experiment." Quarterly Journal of Economics, May 2007, vol. 122, no. 2, pp. 729-773.
Gneezy, Uri, and John A List. "Putting Behavioral Economics to Work: Testing for Gift Exchange in Labor Markets Using Field Experiments." Econometrica, September 2006, vol. 74, no. 5, pp. 1365–1384.
Shearer, Bruce. "Fixed Wages, Piece Rates, and Incentives: Evidence from a Field Experiment." Review of Economic Studies, April 2004, vol. 71, no. 2, pp. 513-534.
Nagel, Daniel, James B. Rebitzer, Seth Sanders, and Lowell J. Taylor. "Monitoring, Motivation, and Management: The Determinants of Opportunistic Behavior in a Field Experiment." American Economic Review, 2002, vol. 92, no. 4, pp. 850-873.
Levitt, Steven D. and John A. List, "What do Laboratory Experiments Measuring Social Preferences Tell Us About the Real World?" Journal of Economic Perspectives, Spring 2007, vol. 21, no. 2, pp. 153-174.
Session 4: Pricing, Advertising, and Consumer Choice - 14 April, afternoon
*Anderson, Eric, and Duncan Simester. "Effects of $9 Endings on Retail Sales: Evidence from Field Experiments." Quantitative Marketing and Economics, March 2003, vol. 1, no. 1, pp. 93-110.
*Gneezy, Uri, and Aldo Rustichini. "A Fine is a Price." Journal of Legal Studies, January 2000, vol. 29, no. 1, pp. 1-18.
*Lewis, Randall, and David Reiley. "Retail Advertising Works! Measuring the Effects of Advertising on Sales via a Controlled Experiment on Yahoo!" Working paper, Yahoo! Research, January 2009. [to be presented in the public lecture at 1800 on April 14]
Drèze, Xavier, Stephen J. Hoch, and Mary E. Purk. "EDLP, Hi-Lo, and Margin Arithmetic." Journal of Marketing, October 1994, vol. 58, no. 4, pp. 16-27. (A field experiment in a supermarket, comparing "everyday low prices" with "hi-lo" promotional prices that fluctuate by product and week.)
Hossain, Tanjim, and Morgan, John. "... Plus Shipping and Handling: Revenue (Non) Equivalence in Experiments on eBay." Advances in Economic Analysis and Policy, 2006. vol. 6, no. 2, article 3.
Wolak, Frank A. "Residential Customer Response to Real-Time Pricing: The Anaheim Critical-Peak Pricing Experiment." Working paper, Stanford University, May 2006.
Ayres, Ian, and Peter Siegelman. "Race and Gender Discrimination in Bargaining for a New Car." American Economic Review, June 1995, vol. 85, no. 3, pp. 304-321.
Goldberg, Pinelopi K. "Dealer Price Discrimination in New Car Purchases: Evidence from the Consumer Expenditure Survey." Journal of Political Economy, June 1996, pp. 622-654. (A study of field data on transaction prices, showing some limitations of Ayres and Siegelman's experiment.)
List, John A. "The Nature and Extent of Discrimination in the Marketplace: Evidence from the Field." Quarterly Journal of Economics, February 2004, vol. 119, no. 1, pp. 49-89. (This study improves on the limitations of Ayres and Siegelman, albeit with lower-valued goods than automobiles.)
Iyengar, Sheena, and Mark Lepper. "When Choice is Demotivating: Can One Desire Too Much of a Good Thing?" Journal of Personality and Social Psychology, 2000, vol. 76, no. 6, 995-1006.
Karlan, Dean, and Jonathan Zinman. "Observing Unobservables: Identifying Information Asymmetries with a Consumer Credit Field Experiment." Working paper, Yale University, December 2008. Forthcoming, Econometrica.
List, John A. "The Behavioralist Meets the Market: Measuring Social Preferences and Reputation Effects in Actual Transactions." Journal of Political Economy, February 2006, vol. 114, no. 1, pp. 1-37.
Session 5: Development Economics (and Program Evaluation) - 16 April
*Duflo, Esther, Michael Kremer, and Jonathan Robinson. "Nudging Farmers to Use Fertilizer: Evidence from Kenya." Working paper, MIT, January 2009.
*Jensen, Robert T. and Nolan Miller, "Giffen Behavior and Subsistence Consumption." American Economic Review, December 2008, vol. 98, no. 4, pp. 1553-77.
Angrist, Joshua, Eric Bettinger, and Michael Kremer. "Long-Term Educational Consequences of Secondary School Vouchers: Evidence from Administrative Records in Colombia." American Economic Review, June 2006, vol. 96, no. 3, pp. 847-862.
Ashraf, Nava, Dean Karlan, and Wesley Yin. "Tying Odysseus to the Mast: Evidence from a Commitment Savings Product in the Philippines." Quarterly Journal of Economics, May 2006, vol. 121, no. 2, pp. 635-672.
Banerjee, Abhijit, Shawn Cole, Esther Duflo, and Leigh Linden. "Remedying Education: Evidence from Two Randomized Experiments in India." Quarterly Journal of Economics, August 2007, vol. 122, no. 3, pp. 1235-1264.
Charness, Gary, and Uri Gneezy. "Incentives to Exercise." Working paper, University of Santa Barbara, December 2008.
Chattopadhyay, Raghabendra, and Esther Duflo. "Women as Policy Makers: Evidence from a Randomized Policy Experiment in India." Econometrica, September 2004, vol. 72, no. 5, pp. 1409-1443.
Duflo, Esther, Rema Hanna, and Stephen Ryan. "Monitoring Works: Getting Teachers to Come to School." Working paper, MIT, November 2007.
Duflo, Esther, Michael Kremer, and Jonathan Robinson. "How High Are Rates of Return to Fertilizer? Evidence from Field Experiments in Kenya." American Economic Review (Papers and Proceedings), May 2008.
Hastings, Justine, and Jeffrey M. Weinstein. "Information, School Choice, and Academic Achievement: Evidence from Two Experiments." Quarterly Journal of Economics, November 2008, vol. 123, no. 4, pp. 1373-1414.
Miguel, Edward, and Michael Kremer. "Worms: Identifying Impacts on Education and Health in the Presence of Treatment Externalities." Econometrica, January 2004, vol. 72, no. 1, pp. 159–217.
Olken, Benjamin A. "Monitoring Corruption: Evidence from a Field Experiment in Indonesia." Journal of Political Economy, April 2007, vol. 115, no. 2, pp. 200-249.
Thornton, Rebecca. "The Demand for, and Impact of, Learning HIV Status." American Economic Review, December 2008, vol. 98, no. 5, pp. 1829-63.
Duflo, Esther. "Field Experiments in Development Economics." Working paper, MIT, January 2006.
Duflo, Esther, Rachel Glennerster, and Michael Kremer, "Using Randomization in Development Economics Research: A Toolkit." Working paper, MIT, January 2007. Forthcoming in Handbook of Development Economics, vol. 4.
Heckman, James J., and Jeffrey A. Smith. "Assessing the Case for Social Experiments." Journal of Economic Perspectives, Spring 1995, vol. 9, no. 2, pp. 85-110.